CSE 212: Programming with Data Structures

Learning Activity: Finding Defects using Tests

The best time to find a defect is while you are "in-phase." This means that before you deliver your software to the customer you want to identify as many errors as possible. It is much more expensive to fix errors if the software has already been delivered to the customer. The two most common methods for finding defects are:

Testing Code From Requirements

Even the best code reviewers may not be able to analyze all the different scenarios that the software will execute within. Testing is a process of demonstrating that specific inputs will result in expected outputs. The selection of inputs can be done systematically. We do not need to test all input combination. Often times we will write code to test your code. Consider a program that was supposed to determine if a year was a leap year. The requirements of a program would include the following:

Based on these requirements, we can write some test cases. Notice that we are not writing test cases based on what the code does. Each test case includes an expected result. If there is an error in the code, we may incorrectly write the test case based on the faulty code. Using the requirements above, we may write the following tests:

Running Test Cases

Notice that each test has a detailed scenario and expected result based on the requirements. If we were given a function called IsLeapYear(int year) we could write inline test code as follows:


var result = isLeapYear(1996);
Console.WriteLine(result);

result = isLeapYear(1900);
Console.WriteLine(result);

result = isLeapYear(2000);
Console.WriteLine(result);

result = isLeapYear(2003);
Console.WriteLine(result);

If anything fails when we run the test code, then we must look manually at the output, then track down where in the code that the test failed.

Instead of printing out the results, test code in C# can use the Trace.Assert function. If the Trace.Assert function fails, then the program will exit and tell you which test (e.g. assert statement) failed. For example:


Trace.Assert(IsLeapYear(1996), "1996 should have been a leap year"); // true
Trace.Assert(!IsLeapYear(1900), "1900 should not have been a leap year"); // false
Trace.Assert(IsLeapYear(2000), "2000 should have been a leap year"); // true
Trace.Assert(!IsLeapYear(2003), "2003 should not have been a leap year"); // false

For more complicated programs, a single test scenario may require you to call multiple functions to properly set up the scenario. For example, if we were testing the enqueue and dequeue functions of a queue class, we might enqueue three numbers and then dequeue the three numbers to ensure that they came out in the correct order. The test code may look like the following:


// Test 1
// Scenario: Ensure that after adding 3 items to the queue, they
// can be removed in the proper order
// Expected Result: 100, 200, 300
Console.WriteLine("Test 1");

var queue = new Queue();
queue.Enqueue(100);
queue.Enqueue(200);
queue.Enqueue(300);

var result = queue.Dequeue();
Trace.Assert(result == 100);

result = queue.Dequeue();
Trace.Assert(result == 200);

result = queue.Dequeue();
Trace.Assert(result == 300);

In addition to finding defects, testing also has the benefit of helping the programmer better understand the requirements of the software. Whether you or another engineer wrote the code, the process of writing test scenarios will increase your understanding of what the software should do.

Testing like this is also critical in the software life-cycle where code needs to constantly be improved. In the Continuous Integration / Continuous Deployment (often abbreviated CI/CD) model of development, operating software or apps can be updated for audiences of millions of users, sometimes many times a day. For this model to work, a key part of the release process has to include rigorous testing that not only checks the functionality of the new features, but also assures regression testing to make sure the new 'fix' hasn't broken something along the way. These tests are automated with assert statements, most-often programmatically generated to get maximum coverage of the code.

Activity Instructions

  1. Open your course repository in VS Code and browse to the week02/learn project. (Note: Do not read through the code first. Instead, proceed with the instructions below).
  2. The SimpleQueue class implements a traditional FIFO queue that has an enqueue and dequeue function. Here are the detailed requirements (which can not be changed):
    1. The enqueue method shall put a new item in the back of the queue
    2. The dequeue method shall remove an item from the front of the queue
    3. If the queue is empty, then the dequeue method shall throw an IndexOutOfRangeException
  3. The code contains three test cases already implemented for you. Run the code and determine if a test has failed. If the test case fails, then you know that there is one or more errors in the function related to the test. If your build crashes, you can see which lines of code the error occurred on.
  4. Find the errors and fix the code. All your test cases should pass when the code is fixed. There are two major errors in the code you were given.

Sample Solution

To see the sample solution, you can uncomment the lines in Program.cs to have it run the SimpleQueueSolution. Then, take a minute to look at the SimpleQueueSolution.cs file to see how the queue was updated.

Key Terms

defect
This is an error in code.
expected result
The result that you expect to receive when you run a test case. The expected result is based on your understanding of the software requirements.
process
The place where software runs. A process has code that is executed and memory used for variables in that code. Multiple processes can run at the same time. Processes can share memory.
requirements
Written description of what software should do.
test cases
Scenarios that are written to test that code behaves per the requirements. A test case will usually have test code unless the procedure is for the user to interact with the software. An expected result is written for each test case.
testing
An activity to demonstrate that the code correctly implements the requirements.

Submission

Make sure to commit and push your changes in your course GitHub repository.

When you have completed all of the learning activities for this week, you will return to I-Learn and submit the associated quiz there.

Other Links: