Testing Smarter: Part 2 – Unit Scope too small

We continue our series on Testing Smarter by looking at common pitfalls around unit scope being either too small or too large.

You can read the first part, where we set the foundations, here: Testing Smarter: Part 1 – Foundations

Scope is too small

An opinion you often hear and see is that a logical unit should be as small as possible: a single class or a method of that class, and, as SOLID describes, that unit should have a single responsibility so that we can test that responsibility. (On a side note, we could have a similar discussion about what constitutes a ‘single responsibility,’ but that is not in the scope of this blog.)
When such a unit that we want to test depends on another class, we need some way to test its single responsibility in isolation, so we use fakes, mocks, and stubs.

Example

Imagine we have an app that supports waste collectors with a couple of functionalities about trash collecting, determining percentages and weights of certain types of garbage, etc.
We’re implementing a service that uses other services to calculate the weight percentages of the different types of garbage within a collection of more waste. There is a different calculator for each type of garbage (it’s their single responsibility), and ours is a percentage calculator based on their calculations.

C#

public class GarbagePercentageCalculator
{
    private readonly IGlassLoadCalculator _glassLoadCalculator;
    private readonly IMetalLoadCalculator _metalLoadCalculator;
    private readonly IPaperLoadCalculator _paperLoadCalculator;
    <...>

    public GarbagePercentages CalculatePercentages(Garbage garbage)
    {
        var glassLoad = _glassLoadCalculator.Calculate(garbage);
        var metalLoad = _metalLoadCalculator.Calculate(garbage);
        var paperLoad = _paperLoadCalculator.Calculate(garbage);

        var glassPercentage = glassLoad / garbage.TotalWeight;
        var metalPercentage = metalLoad / garbage.TotalWeight;
        var paperPercentage = paperLoad / garbage.TotalWeight;

        return new GarbagePercentages(
            glassPercentage, 
            metalPercentage, 
            paperPercentage);
    }
    <...>
}

To mock them in our unit test, we have decoupled the actual dependencies via interfaces.

public class GarbagePercentageCalculatorTests
{
    private readonly Mock<IGlassPercentageCalculator> _glassLoadCalcMock = new(...);
    private readonly Mock<IMetalLoadCalculator> _metalLoadCalcMock = new(...);
    private readonly Mock<IPaperLoadCalculator> _paperLoadCalcMock = new(...);

    private readonly GarbagePercentageCalculator _garbageCalculator;
    ...

    [Fact]
    public void Should_calculate_properties_correctly()
    {
        var input = new Garbage() { ... };

        _glassLoadCalcMock
            .Setup(c => c.Calculate(input)).Returns(15.1235m);
        _metalLoadCalcMock
            .Setup(c => c.Calculate(input)).Returns(33230349.09438m);
        _paperLoadCalcMock
            .Setup(c => c.Calculate(input)).Returns(0.453m);

        var result = _garbageCalculator.CalculatePercentages(input);

        result.Should().BeEquivalentTo(new GarbagePercentages() 
        {
            ...
        });

        _glassLoadCalcMock.Verify(c => c.Calculate(input), Times.Once);
        _metalLoadCalcMock.Verify(c => c.Calculate(input), Times.Once);
        _paperLoadCalcMock.Verify(c => c.Calculate(input), Times.Once);
    }
}

Issues

The question we should be asking ourselves here is the following: What exactly are we testing?

Not only are we testing the expected outcome of our system under test, but we are also testing its implementation. The moment our implementation changes, our test must also change, resulting in a very brittle test. The more we test our implementation, the larger the setup becomes to configure the necessary conditions for that test and the more significant the change necessary to fix the test when changing the implementation.

Let’s say that the GarbageCalculator now needs to call on the PaperLoadCalculator Calculate method twice, even though it should return the same result. Whoops, we broke the test.

Furthermore, not only are we checking our implementation, but we are also testing the signatures of the external dependencies. Consequentially, when any of these signatures change, the test must change with them:

• The MetalLoadCalculator returns a different type than a decimal number? -> Broken test
•  the PaperLoadCalculator now needs two parameters instead of one -> Broken test

This way of testing is not feasible for long-term development, especially when 1000s of these tests exist.

Solution: test the result and not the implementation!

We could easily posit that the calculations of the different load calculators are part of our logical unit. Thus, we should not mock them and inject them directly into our system under test.
We could do away with the interfaces and let our aggregator create the other calculators or make them static helper classes so that the test does not need to change when the constructors of these calculators change. Still, these changes are all highly dependent on the context; this is another topic that is out of the blog’s scope.

public class GarbagePercentageCalculatorTests
{
    private readonly IGlassPercentageCalculator _glassLoadCalculator = new(...);
    private readonly IMetalLoadCalculator _metalLoadCalculator = new(...);
    private readonly IPaperLoadCalculator _paperLoadCalculator = new(...);

    private readonly MyGarbagePercentageCalculator _garbageCalculator;
    ...

    [Fact]
    public void Should_calculate_properties_correctly()
    {
        var input = new Garbage() { ... };

        var result = _garbageCalculator.CalculatePercentages(input);

        result.Should().BeEquivalentTo(new GarbagePercentages() 
        {
            ...
        });
    }
}

Now our test only needs to change when our method’s input or output changes. Fewer reasons for the test to change means less time needed to change the code, which means faster development times!

In summary, our test was brittle because the scope we used for our logical unit was too small. This resulted in an attempt to isolate our system under test from its dependencies, which should have been part of the test, exacerbating that brittleness.
A good rule of thumb would be to use mocks sparingly. More mocks equals a more extensive test setup and more reasons to change. Of course, this might not be possible when dealing with external dependencies like a database or third-party API. We will cover these situations in a later example.

Other posts in this series:

• Testing Smarter: Part 1 – Foundations
• Testing Smarter: Part 3 – Unit scope too large
• Testing Smarter: Part 4 – Boundary Testing Pitfalls
• Testing Smarter: Part 5 – Conclusions