CSE 270 | Learning Activities

W06 Assignment: Unit Testing Techniques + Coverage Tools

Overview

Unit testing can sometimes involve being creative with testing techniques. Inside the cse270-v15.zip file is the tests folder which you would have downloaded in the previous assignment. You will see two files in this folder that we will look at together.

examples.py
test_examples.py

Let's look at each example individually.

Testing Side Effects

Sometimes a function that you call has side effects, such as modifying an object that is passed in as a parameter. When this happens, we can set up the specific data we want to test and evaluate the results of calling the function with that data. Here is an example:

def find_and_replace(some_list, find, replace):
    # Finds and replaces the items passed in
    # Raises ValueError when the item is not found
    some_list[some_list.index(find)] = replace

In this example, we can pass a list in as a parameter and the item in the list "find" will be replaced by the data in "replace". To test this, we merely create our own list, and pass in the data to the function, then evaluate if the list changed in the way we expected. Because find_and_replace doesn't return a value, we don't need to evaluate the result returned from the function.

def test_find_and_replace():
  mylist = ['apples','oranges']
  find_and_replace(mylist, 'apples', 'bananas')
  assert mylist[0] == 'bananas'

Testing For Raised Exceptions

In the previous example, we might attempt to replace an item in the list that doesn't exist. When this happens an error is raised by Python. Pytest gives us the ability to test for conditions that result in an error. We use the pytest.raises(error) function as shown below. By calling the function under test within the with block, pytest will evaluate if the function raised the error specified.

def test_find_and_replace_error()
  mylist = ['apples','oranges']
  with pytest.raises(ValueError):
    find_and_replace(mylist, 'bananas', 'pears')

Testing Random Outcomes

Sometimes the outcome of a function call cannot be known in advance. Here is an example found in examples.py:

some_list = ['alpha','beta','gamma','delta']      
def get_random_item():
  return random.choice(some_list)

In this scenario you have a few choices.

You can check to see if the choice returned falls within the range of available options
You can mock the random.choice method so it returns an object that you define

In the example below, the first method is chosen and after calling the function, we assert that the choice was found in the list of available choices. We import the list of available choices directly from the module we are testing. We could also set up our own list of choices, but then we would have to maintain that list separately from the list inside the program.

This example (as well as all other testing examples) is found in the file test_examples.py

import pytest
from examples import (
  get_random_item,
  gather_input,
  get_data_from_file,
  get_user_id_1,
  some_list
  )
## test for a choice that falls within the list "some_list" from the example
def test_get_random_item_no_mock():
  choice = get_random_item()
  assert choice in some_list

Our second choice is to use a mock to simulate the behavoir of the random.choice function, which is used by our function to make the choice. We can set up the random.choice function to always return a particular answer. In this way we can test as well.

You may wonder where the mocker variable comes from. This is a mock object that is automatically injected by the pytest framework. The mock object has a lot of useful features like "patch" which sets up a new function in place of another function for testing purposes.

The mock object also has a feature that lets us test how many times a particular function was called. The line mock_choice.assert_called_once() checks to see that the random.choice() method was only called one time in the function under test.

# Test for get_random_item function
def test_get_random_item(mocker):
  # Patch the random.choice method with a specific return value "alpha"
  mock_choice = mocker.patch("random.choice",return_value="alpha")
  # assert that the choice returned is "alpha"
  assert get_random_item() == 'alpha'
  # assert that the choice method was called once
  mock_choice.assert_called_once()

Testing a function that reads from the console

If we have a function that requires input from the user, this is a problem, because we need our tests to run automatically without user intervention. See the example code below:

def gather_input(prompt):
    return input(prompt)

How can we achieve this? Again - we can use the "patch" feature of the mock object to patch the "input" method which is used to gather user information. We can create a separate function that returns a known response (like "Test input") instead of waiting for user input. Notice that "builtins.input" refers to the input() function that is built in to the python language.

# Test for gather_input function
def test_gather_input(mocker):
  # Mock the input function
  mock_input = mocker.patch("builtins.input", return_value="Test input")
  # Call the function under test
  result = gather_input("Enter input: ")
  # Assert the result
  assert result == "Test input"
  # Assert that the input function was called with the correct prompt
  mock_input.assert_called_once_with("Enter input: ")

Testing functions that read from files

Some of your functions may read data from the file system. This is once again problematic for automatic testing, because there is no guarantee that the file you need to read from is where you think it is. Consider the simple example below that reads the contents of a filename passed in.

def get_data_from_file(filename):
  contents = ""
  with open(filename) as myfile:
  contents += myfile.readline()
  return contents

To test this program, we have a couple of options.

Create a temporary file in the setup of the test, the read from that file to test.
Use the patching capability of mocker to update the built-in open() function so that it returns a known sequence of data.

Here's an example of the first method. The parameter tmp_path passed in to the test function in this case is a special function called a "fixture" that helps Python determine an appropriate location to save a temporary file. The tmp_path fixture is built in to Pytest and is used for just this purpose. To learn more about the tmp_path fixture, you can look here. https://docs.pytest.org/en/7.0.x/how-to/tmp_path.html

# Tests getting data from a file creating a temp file for testing
def test_get_data_from_file(tmp_path):
  # Create a temporary file with test data
  test_data = "file contents"
  file_path = tmp_path / "test.txt"
  with open(file_path, "w") as f:
  f.write(test_data)
  # Call the function under test
  result = get_data_from_file(file_path)
  # Assert the result
  assert result == test_data

If you don't want to actually create a file, you can use the patching mechanism we have explored previously to override the built-in open() method and return a predetermined response.

# Test for get_data_from_file function
def test_get_data_from_file(mocker):
  # Mock the open function
  mock_open = mocker.patch("builtins.open", mocker.mock_open(read_data="file contents"))
  # Call the function under test
  result = get_data_from_file("test.txt")
  # Assert the result
  assert result == "file contents"
  # Assert that the open function was called with the correct filename
  mock_open.assert_called_once_with("test.txt")

Testing a function that talks to a database

In the example code you we have used the psycoph2 third party module to connect to a PostgreSQL database and get information from a table called USERS.

def get_user_id_1():
  # Connect to PostgreSQL database
  conn = psycopg2.connect(
    dbname="mydb",
    user="username",
    password="password",
    host="127.0.0.1",
    port="5432"
  )
  # Create a cursor object
  cur = conn.cursor()
  try:
    # Execute SQL query to fetch user at row 1
    cur.execute("SELECT * FROM USERS WHERE ID=1")
    # Fetch the user data
    user = cur.fetchone()
    # Commit the transaction
    conn.commit()
    return user
  except (Exception, psycopg2.DatabaseError) as error:
    print("Error fetching user:", error)
    # Rollback the transaction in case of error
    conn.rollback()
  finally:
    # Close the cursor and connection
    cur.close()
    conn.close()

In this function there is a lot going on - the function creates a connection, gets a cursor, executes the query using the cursor, fetches some data, commits the transaction and closes both the cursor and the connection.

When testing a function that gets data from a database, we again have a couple of options.

We can mock the response received from our database call without actually calling the database
We can read data from a test database that has been set up expressly for that purpose

Let's review the example test. In this example, we created our own "fixture" which takes the mocker and mocks the call to "psycopg2.connect". When we pass this to our test_get_user_id_1 function, Pytest looks for a fixture with the same name and injects that function into our code. With the mock connect object, we can get a mock cursor as well. We can mock the call to "fetchone" which gets a single line of data from the database. We can then call the method get_user_id_1() and inspect the return value as well as all the interactions with the mock that we are expecting, like commiting the transaction and closing the connection.

# Set up a mock object for the database connection
@pytest.fixture
def mock_psycopg2_connect(mocker):
  # Patch the psycopg2.connect method
  mock_connect = mocker.patch('psycopg2.connect')
  # Return the mock object
  return mock_connect
# Test for get_user_id_1 function
def test_get_user_id_1(mock_psycopg2_connect):
  # Get a mocked version of the connection for later
  mock_connect = mock_psycopg2_connect.return_value
  # Mock the cursor and its methods
  mock_cursor = mock_psycopg2_connect.return_value.cursor.return_value
  mock_cursor.fetchone.return_value = 'test_user'
  # Call the function under test
  user = get_user_id_1()
  # Assert the result
  assert user == 'test_user'
  # Assert the database connection and cursor were called as expected
  mock_psycopg2_connect.assert_called_once_with(
    dbname="mydb",
    user="username",
    password="password",
    host="127.0.0.1",
    port="5432"
  )
  #Asset that each of the functions below were called once as expected
  mock_cursor.execute.assert_called_once_with("SELECT * FROM USERS WHERE ID=1")
  mock_cursor.close.assert_called_once()
  mock_connect.commit.assert_called_once()
  mock_connect.close.assert_called_once()

Setting up a test database then connecting to it will only require updating the connection information then calling the function get_user_id_1 and then inspecting the result for well-known values. Because the examples.py program doesn't have an actual database to connect to, we will not demonstrate that here.

Testing framework variations

Pytest is a popular third party framework used for doing testing. The examples shown above make use of capabilities and featues thare are included with Pytest. Python comes with its own testing framework called "unittest". Unittest has many features similar to Pytest, but they work a little differently. Your choice of testing framework will require you to learn the syntax and methods that are particular to each framework you use.

Using coverage.py to determine code coverage

As part of today's assignment, we will use the code coverage tool coverage.py to assess our coverage of the functions under test. You must first install the coverage program the same way you would have installed pytest earlier. We also need the psycopg2 and the pytest-mock packages which are used in the examples.Open a terminal window and execute the following:

For Windows:

pip install coverage psycopg2-binary pytest-mock

For Mac:

pip3 install coverage psycopg2-binary pytest-mock

To determine how much coverage your program has, you can run the program in this way.

From your terminal window, make sure you are in the tests folder by typing the following:

cd tests

After making your tests folder the current directory in your terminal, you can generate a coverage report for our examples.py file by typing the following at the terminal:

coverage run -m pytest test_examples.py

You will see test output from pytest because it will have run the test_examples.py file to gather information.

Now, to see the coverage report type the following at the terminal:

coverage report -m

This will give you a text-based report that gives you a coverage percentage and indicates which lines of code have not been covered by a test. You will notice that in examples.py, lines 40-43 are not covered by any test in our test_examples.py file. This is the code that checks for error conditions when connecting to the database.

Instructions

In this assignment you will apply what you have learned by writing a tests for a program that uses many of the concepts previously explained.

Update the file test_build_sentences.py found in the tests folder of the cse270-v15 project. Remove the word pass and add the proper code to test the actual function. This file will test each of the functions in the build_sentences.py file except the main() function.
After your tests pass, run the coverage tool to demonstrate that your tests cover all of the code paths except the main() function and the call to main(). This means your coverage report should show everything tested except lines 106-110 and line 113.

What to turn in?

Return to Canvas and submit two artifacts for this assignment.

The test_build_sentences.py file that tests build_sentences.py
A screenshot of your run of coverage.py showing the coverage report with the missing lines.

Submission

Useful Links:

Return to: Week Overview | Course Home | Canvas