Test Fixtures and Mocking

Better Code, Better Science: Chapter 4, Part 4

This is a section from the open-source living textbook Better Code, Better Science, which is being released in sections on Substack. The entire book can be accessed here and the Github repository is here. This material is released under CC-BY-NC.

Also: This week I will be putting out one new post every day, in honor of the Neurohackademy workshop going on this week.

Test fixtures

Sometimes we need to use a the same data for multiple tests. Rather than duplicating potentially time-consuming processes across each of the tests, it is often preferable to create a single instance of the object that can be used across multiple tests, which is known as a test fixture. This also helps maintain isolation between tests, since the order of tests shouldn't matter if an appropriate fixture is generated as soon as it's needed.

For our example above, it's likely that we will need to reuse the list of pubmed IDs from the search to perform various tests on the subsequent functions. We can create a single version of this list of IDs by creating a fixture. In the pytest framework we do this using a special Python operator called a decorator, which is denoted by the symbol @ as a prefix. A decorator is function that takes another function as input, modifies its functionality, and returns another function; you don't need to understand in detail how decorators work for this particular usage. To refactor our tests above, we would first create the fixture by decorating the function that generates the fixture with the @pytest.fixture decorator, setting the scope variable to "session" so that the fixture is only generated once within the session:

@pytest.fixture(scope="session")
def ids():
    query = "friston-k AND 'free energy'"
    ids = get_PubmedIDs_for_query(query)
    return ids

We can then refactor our tests for a valid query to use the fixture by passing it as an argument to the test function:

def test_get_PubmedIDs_for_query_check_valid(ids):
    assert isinstance(ids, list)
    assert len(ids) > 0

The result is the same, but we now have a set of ids that we can reuse in subsequent tests, so that we don't have to make repeated queries. It's important to note while using a session-scoped fixture: If any of the subsequent tests modify the fixture, those modifications will persist, which will break the isolation between tests. We could prevent this by removing the scope="session" argument, which would then default to the standard scope which is within a specific function. If you wish to use session-scoped fixtures and need to modify them within the test function, then it is best to first create a copy of the fixture object (e.g. my_ids = ids.copy()) so that the global fixture object won't be modified.

Mocking

Sometimes tests require infrastructure that is outside of the control of the tester. In the example above, we are assuming that the Pubmed API is working correctly for our tests to run; if we were to try to run these tests without an internet connection, they would fail. In other cases, code may rely upon a database system that may or may not exist on a particular system. In these cases, we can create a mock object that can stand in for and simulate the behavior of the system that the code needs to interact with.

In our example, we want to create a mock response that looks sufficiently like a response from the real API to pass our tests. Using pytest's monkeypatch fixture, we can temporarily replace the real requests.get function with our own fake function that returns a predictable, controlled response. We first need to create a class that can replace the requests.get() call in get_PubmedIDs_for_query(), replacing it with a mock version that outputs a fixed simulacrum of an API response via its .json() method.

class MockPubmedResponse:
    status_code = 200

    def json():
        return {
            'header': {'type': 'esearch', 'version': '0.3'},
            'esearchresult': {
                'count': '2',
                'retmax': '20',
                'retstart': '0',
                'idlist': ['39312494', '39089179']
            }
        }

We now insert this mock response for the standard requests.get() call within the test. In my initial attempt, I created created a fixture based on the mocked response and then tested that fixture:

@pytest.fixture
def ids_mocked(monkeypatch):

    def mock_get(*args, **kwargs):
        return MockPubmedResponse()

    # apply the monkeypatch for requests.get to mock_get
    monkeypatch.setattr(requests, "get", mock_get)

    query = "friston-k AND 'free energy'"
    ids = get_PubmedIDs_for_query(query)
    return ids

def test_get_PubmedIDs_for_query_check_valid_mocked(ids_mocked):
    assert isinstance(ids_mocked, list)
    assert len(ids_mocked) == 2

Turning off my network connection shows that the mocked test passes, while the tests that require connecting to the actual API fail. However, my usual code review (using Google's Gemini 2.5 Pro) identified a problem with this fixture: it conflates the setup (creating the mock API) with the execution of the function that uses the mock API. A better approach (recommended by Gemini) is move the function execution out of the fixture and into the test:

# Fixture ONLY does the setup (the mocking)
@pytest.fixture
def mock_pubmed_api(monkeypatch):

    class MockPubmedResponse:
        status_code = 200
        def json(self):
            return {
                'header': {'type': 'esearch', 'version': '0.3'},
                'esearchresult': {
                    'count': '2',
                    'retmax': '20',
                    'retstart': '0',
                    'idlist': ['39312494', '39089179']
                }
            }

    def mock_get(*args, **kwargs):
        return MockPubmedResponse()

    # Apply the monkeypatch for requests.get to mock_get
    monkeypatch.setattr(requests, "get", mock_get)

# The test requests the setup, then performs the action and assertion.
def test_get_PubmedIDs_for_query_check_valid_mocked(mock_pubmed_api):
    # Action: Call the function under test
    query = "friston-k AND 'free energy'"
    ids = get_PubmedIDs_for_query(query)

    # Assertion: Check the result
    assert isinstance(ids, list)
    assert len(ids) == 2

Note that while mocking can be useful for testing specific components by saving time and increasing robustness, integration tests and smoke tests should usually be run without mocking, in order to catch any errors that arise through interaction with the relevant components that are being mocked. In fact, it's always a good idea to have tests that specifically assess the usage of the external service and the system's response to failures in that service (e.g. by using features of the testing framework that allow one to shut down access to the network).

In the next installment I will discuss parameterized testing.