CSE 370: Software Engineering Principles

W01 Learning Activity: The Software Development Life Cycle

Overview

This activity will help you learn about the major components of the software development life cycle (SDLC) that will be discussed throughout this course. It also highlights aspects of software development methodologies.

Preparation Material

Software methodologies and the software development life cycle (SDLC) are closely related concepts in the field of software engineering. The SDLC represents the various phases involved in the development and maintenance of a software product, while software methodologies provide structured approaches to managing and executing these phases.

It is also important to realize that compared to other kinds of engineering, software engineering is a relatively young and constantly evolving field. For many of the topics in this lesson, and throughout the course, there is no consensus established by a governing body. Instead, each company is doing their best to follow good practices and figure out the best approach they can. Despite the lack of a single established pattern, there are principles and good practices (as well as bad practices) that have been vetted and shared throughout the community.

Throughout this course, you will learn about these established principles and current best practices around them. You will also learn how to analyze situations and projects through the lens of these principles and practices, so that you will be prepared for future developments in the field.

The Software Development Life Cycle

Every software product goes through phases during its life cycle. Regardless of the organization or structure, each project will include at least some version of the following steps:

• Deciding what should be built.
• Planning how to build it.
• Building it.
• Making sure it works and does what it should.
• Delivering it to the people that will use it.
• Fixing and enhancing it.

Depending on the process, these steps may have slightly different names. They may be done more sequentially or sometimes more iteratively. There may be more formality or sometimes more flexibility. The important thing to remember is that these concepts need to be represented in some fashion, and that no one process is best for all organizations and businesses.

These steps are often summarized into the following phases (shown here along with the week that it will be discussed in this course):

1. Requirements Elicitation (Week 02): Gathering and documenting the functional and non-functional requirements.
2. Design (Week 03): Creating the architecture and design of the system.
3. Implementation: Writing the code according to the design specifications.
4. Testing and Verification (Week 04): Verifying that the software works as intended and is free of defects.
5. Release and Maintenance (Week 05): Releasing the software to users and providing ongoing support and updates to the software.

Throughout this course, you will learn about each of these components and various issues that can arise because of decisions made in various steps of the process. With the exception of implementation, there is one unit in this course for each major phase of the SDLC. Even though it is at the core of the software development process, this course will not specifically discuss the implementation phase, because that has been the focus of nearly all the other courses have taken. Instead, this course focuses on all the other components of the SDLC.

DevOps

Historically, software engineers did all of the work to develop a product and then at the end, they would more or less hand it off to an Information Technology (IT) operations group who would deploy the solution. In recent years, companies have found that there are great benefits to integrating the developers and operations people through the entire software life cycle. The term to describe the integration of developers (Dev) and IT operations (Ops) is DevOps.

DevOps is a set of practices, cultural philosophies, and tools that aims to improve collaboration and integration between software development and IT operations teams. The goal of DevOps is to shorten the software development lifecycle and deliver high-quality software continuously. By fostering a collaborative environment, DevOps seeks to automate processes and enhance the ability to deliver features, fixes, and updates frequently and reliably.

Key Principles of DevOps

• Culture of Collaboration: DevOps promotes a culture where development and operations teams work closely together, breaking down traditional silos. This collaboration fosters better communication, shared responsibilities, and a focus on common goals.
• Automation: Automation is a cornerstone of DevOps, aimed at streamlining repetitive tasks such as code integration, testing, deployment, and infrastructure management. Automation tools reduce human error, improve efficiency, and enable faster delivery cycles.
• Continuous Integration and Continuous Delivery (CI/CD):
  • Continuous Integration (CI): Developers regularly merge their code changes into a shared repository, where automated builds and tests are run. This practice helps detect issues early and ensures code quality.
  • Continuous Delivery (CD): Extends CI by automatically deploying code changes to a staging or production environment after passing the necessary tests. Continuous Deployment takes this a step further by deploying every change that passes automated tests to production.
• Infrastructure as Code (IaC): IaC is the practice of managing and provisioning computing infrastructure through machine-readable configuration files rather than physical hardware configuration or interactive configuration tools. This allows for version control, automated provisioning, and consistent environments.
• Monitoring and Logging: Continuous monitoring and logging provide insights into the performance and health of applications and infrastructure. This data helps identify issues, optimize performance, and ensure that systems are running as expected.
• Security (DevSecOps): Integrating security practices into the DevOps process, known as DevSecOps, ensures that security is considered at every stage of the development life cycle. This includes automated security testing and compliance checks.

The stages of DevOps are similar to the traditional SDLC phases. However, they are meant to highlight the more iterative and connected nature of the stages, and particularly the integration of the development and operations groups into a single workflow. These stages of DevOps are often listed as:

1. Plan: This phase consists of defining and planning what will be built. It involves gathering the requirements as well as the business metrics and value. At some level, it includes the planning of each of the other stages, including a release plan and security requirements.
2. Create: This phase includes the the design and coding of the software itself.
3. Verify: This phase ensures that the software meets the necessary requirements and includes acceptance testing, regression testing, performance testing, and security vulnerability analysis.
4. Package: This phase consists of all activities related to preparing the software for deployment such as approvals, packaging, and staging releases.
5. Release: This phase includes coordinating the release, deploying or promoting the software, and handling fallbacks and recovery.
6. Configure: This phase falls more directly on the operations side of the spectrum and includes any infrastructure provisioning and configuration necessary, such as storage, database, or networking.
7. Monitor: This phase allows the organization to identify issues of a release and its impact on end-users, including performance of the infrastructure, end-user experience, and production metrics.

The following figure shows the stages of the DevOps process, shown in an infinite loop to highlight the ongoing and connected nature of the stages:

Software Development Methodologies

Software methodologies are systematic approaches used to manage and execute the phases of the SDLC. They provide guidelines, practices, and processes that help teams effectively develop software.

Traditional Processes

Historically, software projects have followed a sequential approach to the steps of the SDLC where each step was completed and informed the work of the next step.

Waterfall

The Waterfall process defines that you should follow each of the phases of the SDLC in a linear, sequential approach. The name waterfall captures how each step flows into the next one like water flowing downward.

The typical steps of the waterfall process are:

• Requirements Analysis: In this phase, the project requirements are gathered, analyzed, and documented. This involves detailed discussions with stakeholders to understand their needs and expectations. The outcome is a requirements specification document that serves as a guideline for the subsequent phases.
• System Design: Based on the requirements specification, the system architecture and design are created. This includes defining the system's hardware and software requirements, overall system architecture, data models, and detailed design specifications. This phase results in design documents that provide a blueprint for implementation.
• Implementation: In the implementation phase, the actual source code is written based on the design documents. Developers translate the system design into functional software components. This phase typically involves coding, unit testing, and integration of the individual modules.
• Integration and Testing: After the implementation, the software is integrated and tested as a whole system. This phase includes system testing, integration testing, and acceptance testing to ensure that the software meets the specified requirements and is free of defects. Bugs are identified and fixed during this phase.
• Deployment: Once the software passes all testing phases, it is deployed to the production environment. This involves installation, configuration, and making the system operational for end-users. This phase may also include user training and documentation.
• Maintenance: The maintenance phase involves making changes, corrections, and improvements after the software has been deployed. This includes fixing any issues that are discovered, making updates, and enhancing the system as needed over time.

The waterfall process is sequential, where each phase is completed before going on to the next step. It places a strong emphasis on documentation to describe the decisions made in each phase. Because of the clear deliverables at each step, the waterfall process provides milestones that are easy to observe. In addition, industries that have a high degree of regulation, such as government contracts, or those where a system is very well defined and with few unknowns, benefit from the structure and clear definitions of the process.

The main disadvantage of the Waterfall process is that it is not very flexible. As new ideas or discoveries occur later in the process it is very difficult to adapt. One of the perceived benefits of Waterfall is that there is a sense of schedule and deadlines for when products will be delivered. But in practice, unknowns that arise later in the process almost always delay this schedule, sometimes in significant ways.

The Waterfall process has received extensive criticism in recent years, but it is important to recognize that the steps of the process are in fact the phases that must be accomplished in some form. The improvements that have been made are in finding ways to walk through these phases in more rapid, iterative, or integrated ways.

The V-Model

The V-Model, also known as the Verification and Validation Model, is an extension of the traditional waterfall model. It emphasizes the importance of testing at each stage of development. It specifies a direct testing phase that corresponds to each development phase, forming a V shape when depicted graphically.

The following figure shows the typical phases, although they may be known by slightly different names or be subdivided in different ways:

Notice that each development phase has a corresponding testing phase that verifies it, such as Requirements Analysis being verified by User Acceptance testing.

One of the great strengths of a V-Model approach is the clear documentation produced at each step in the development process that contains sufficient detail to verify that it was done correctly. Also, verification is not seen as a single event that looks only at code quality or at user acceptance testing, but rather a systematic review of all of the work and expectations of the development phase, from low-level to high-level.

Agile Processes

In 2001, a group of leading Software Engineering thinkers met to discuss the state of software engineering as a discipline and find common ground. What emerged was a document they termed the Agile Manifesto.

The manifesto reads:

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

As stated in the manifesto, the group recognized value in processes, tools, and plans, but they also saw the need for the software process to be more adaptive and responsive to the needs of the customer.

They included twelve principles behind the manifesto:

• Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
• Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
• Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
• Business people and developers must work together daily throughout the project.
• Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
• The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
• Working software is the primary measure of progress.
• Agile processes promote sustainable development.
• The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
• Continuous attention to technical excellence and good design enhances agility.
• Simplicity—the art of maximizing the amount of work not done—is essential.
• The best architectures, requirements, and designs emerge from self-organizing teams.
• At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Scrum

One of the most popular Agile methodologies is Scrum. Many software companies use principles of Scrum, even if they don't use every aspect of it exactly as prescribed. Scrum emphasizes iterative progress, collaboration, and adapting to changing requirements. The main approach is to break projects into small, manageable pieces that are developed and delivered in time-boxed iterations called sprints.

A sprint is a short period of time, such as two weeks. At the beginning of the sprint, the team determines what should be built and delivered during that period. Then, during the sprint, the team collaborates extensively, including a daily stand-up meeting, where each person shares a brief update of their work and plans. At the end of the sprint, the team discusses what went well and what could be improved in the next iteration.

There are a few key artifacts, roles, and events that define the Scrum methodology.

Artifacts

• Product Backlog: An ordered list of everything that might be needed in the overall product. It is dynamic and evolves as the product and environment change.
• Sprint Backlog: A subset of the product backlog items selected for a specific sprint, along with a plan for delivering them.
• Product Increment: The sum of all the product backlog items completed during a sprint combined with the increments of all previous sprints. In other words, this is the next version of the working product that is ready to be released at the end of the sprint. An increment must be in a usable condition and meet the team's definition of done.

Roles

• Product Owner: Represents the stakeholders and the voice of the customer. They are responsible for defining the features of the product and prioritizing the product backlog. The product owner doesn't get to prescribe what must be completed during the sprint, because the team is responsible to estimate how long each time may take (and therefore the scope of the sprint), but the product owner defines the priority of what should be developed next.
• Scrum Master: Facilitates the Scrum process, ensures the team adheres to Scrum practices, and removes any obstacles that impede the team's progress. The Scrum Master is not the product manager or development lead. Instead, their role is to make sure the process is followed correctly.
• Development Team: A cross-functional group of professionals responsible for delivering the product increments. The team is self-organizing and decides how best to accomplish their work.

Events

• Sprint: A time-boxed iteration, typically 1-4 weeks long, during which a potentially shippable product increment is created. Sprints have consistent durations throughout a project.
• Sprint Planning: A meeting held at the beginning of a sprint, where the Scrum team plans the work for the upcoming sprint. The team selects items from the product backlog and defines a sprint goal. It is important that each task in the backlog has been broken down into small enough pieces that each task could be completed in less than a single day.
• Daily Scrum (Stand-up): A 15-minute time-boxed meeting held daily, where team members synchronize their work and plan for the next 24 hours. This meeting is called a stand-up meeting because it is common to require members to actually stand for the meeting, thus helping keep it short and focused. In these meetings, each team member shares an update on what they accomplished the previous day and what they plan to complete before the next meeting. They also communicate if they have anything blocking them from their current tasks.
• Sprint Review: Held at the end of the sprint to inspect the increment and adapt the product backlog if needed. The team demonstrates what they have built to stakeholders.
• Sprint Retrospective: A meeting for the Scrum team to reflect on the past sprint and identify improvements for the next sprint. They typically focus on two questions: "What did the team do well in the previous sprint?" and "What will the team improve for the next sprint?"

Because of the short nature of each sprint, Scrum processes are flexible and can adapt to changing requirements and feedback. The daily connection point of the stand-up meeting promotes communication and collaboration among team members. The retrospective meetings facilitate continuous improvement of the team and the process.

Kanban

Another popular agile methodology is Kanban. Like Scrum, many software companies use elements of Kanban, even if they do not follow every aspect it prescribes.

Kanban is a visual workflow management method used in software engineering and other fields to optimize the flow of work and improve efficiency. Originating from the Japanese manufacturing industry, Kanban focuses on continuous delivery without overburdening the development team. It uses visual cues to signal the various stages of the workflow, ensuring that work is efficiently managed and bottlenecks are identified and resolved.

The Kanban workflow is visualized using a Kanban board, sometimes referred to as an information incubator, which consists of columns representing different stages of the process (for example, Backlog, To Do, In Progress, Done). Cards representing work items move through these columns as they progress. Sometimes these cards are electronic, but often they are physical sticky notes that people move along a wall in the team's work area. In addition to the columns, occasionally "swimlanes" are created as horizontal rows for different types of roles or teams.

One of the key principles of Kanban is to limit the Work in Progress at any given time to avoid overloading the team so they can stay focused on the task at hand.

Compared to Scrum, Kanban is more generic and flexible because it does not provide the specific roles, events, and iterations in the same way. Kanban focuses on managing and optimizing the flow of work in a continuous process, whereas Scrum organizes the work into time-boxed iterations.

ScrumBan

Many companies use principles from both Scrum and Kanban in a way that is sometimes referred to as ScrumBan. Essentially, they follow the processes and meetings of Scrum, but use a KanBan board to track the individual tasks of the sprint. It is common for people to move the cards on the KanBan board during their daily standup Scrum meeting.

Agile and the Future

It has now been over 20 years since the Agile Manifesto was released. Since that time, there have been many flavors of Agile approaches that have become very popular. Nearly all software development processes use many Agile principles and processes in some way.

Agile processes have proven to help many companies reliably deliver working software.

Occasionally, you may also see articles that say Agile is dying or studies that show that agile projects have just as much trouble as traditional ones at delivering software on-time. As you will see throughout this course, creating a novel software system is an incredibly difficult process to define and manage, regardless of the process used to manage it.

If it is true that agile may be dying in some respects, one of the reasons could be that humans have overcomplicated the Agile processes so much that they have become rigid. In our community's desire to standardize on best practices, we have created all kinds of processes, trainings, and vocabulary. If we are not careful, we lose the spirit of exactly what these methods were designed for. In dismay, one software engineer said, "Just ask the customer... I don't need to fill out the forms, I don't need to follow all the arcane vocabulary. The whole point was to be agile." To be clear, the processes and vocabulary are very important, but to take full advantage of what agile processes can offer, you must be careful not to become to rigid in the application.

Which Process to Choose?

Software methodologies provide structured approaches for executing the phases of the SDLC. For example, the Waterfall model provides a clear sequence of phases, while Agile methodologies like Scrum and Kanban offer iterative and incremental approaches.

Different methodologies offer varying levels of flexibility. Waterfall is rigid and best suited for projects with well-defined requirements, whereas Agile methodologies are more adaptable to changes, making them suitable for projects with evolving requirements.

The specific choice of methodology depends on various factors, including:

• Project Requirements: Stable requirements might benefit from a Waterfall approach, while projects with dynamic requirements might be better suited for Agile methodologies.
• Team Experience: Teams experienced with Agile practices might prefer Scrum or Kanban, while those accustomed to traditional approaches might lean towards Waterfall or V-Model.
• Project Size and Complexity: Large, complex projects might require the structured approach of Waterfall or V-Model, while smaller, less complex projects might benefit from the flexibility of Agile methodologies.

Submission

After you are comfortable with these topics, return to Canvas to take the associated quiz.

Other Links:

• Return to: Week Overview | Course Home

ChatGPT assisted in the creation of this learning material.