Testing Smarter: 4 – Boundary Testing Pitfalls

December 1, 2023 — Zinderlabs

Similar to the previous example, we have an endpoint that should create a new book, but here the business logic is much more straightforward.

Example

We will start with the controller.

[ApiController]
[Route("api/[controller]")]
public class BooksController : Controller
{
    private readonly IBooksService _booksService
    public BooksController(IBooksService booksService) => _booksService = booksService;

    [HttpPost]
    public async Task<ApiResponse<int>> CreateBook([FromBody] CreateBookRequest request)
    {
        var id = await _booksService.CreateBook(request);
        return new ApiResponse(id);
    }
}

And its test

public class BooksControllerTests
{
    private readonly Mock<IBooksService> _booksServiceMock = new();
    private readonly BooksController _booksController;

    public BooksControllerTests() => 
        _booksController = new(_booksServiceMock.Object);

    [Fact]
    public async Task Service_should_be_called_to_create_book() 
    {
        var request = new CreateBookRequest() { ... };

        _booksServiceMock
            .Setup(s => s.CreateBook(request))
            .ReturnsAsync(1);

        var result = await _booksController.CreateBook(request);

        result.Should().Be(new ApiResponse(1));
    }
}

Next up is the BookService.

public class BooksService
{
    private readonly IBooksRepository _booksRepo;
    private readonly IDomainEventPublisher _messagePublisher;

    public SomeService(IBooksRepository booksRepo, IDomainEventPublisher publisher)
    {
        _booksRepo = booksRepo;
        _messagePublisher = publisher;
    }

    public async Task<int> CreateBook(CreateBookRequest request)
    {
        var book = new Book(request.name, request.date);
        var id = await _booksRepo.Create(book);
        await _messagePublisher.Publish(new BookCreatedEvent(id, book));

        return id.Value;
    }
}

public class BooksServiceTests
{
    private readonly Mock<IBooksRepository> _booksRepoMock = new();
    private readonly Mock<IDomainEventPublisher> _messagePublisherMock = new();

    private readonly BooksService _booksService;

    public BooksServiceTests() _booksService = new(_booksRepoMock.Object, _messagePublisherMock.Object)

    [Fact]
    public async Task Book_should_be_created()
    {
        Book actualBook = null;
        _booksRepoMock.Setup(r => r.Create(It.IsAny<Book>()))
            .Callback((Book b) => actualBook = b)
            .ReturnsAsync(new Id(1));

        var request = new CreateBookRequest() { ... };
        var id = await _booksService.Create(request);

        id.Should().Be(new Id(1));
        actualBook.Should().Be(...);
    }

    [Fact]
    public async Task An_event_should_be_sent_on_book_creation()
    {
        _booksRepoMock.Setup(r => r.Create(It.IsAny<Book>())).ReturnsAsync(new Id(1));

        BookCreatedEvent sentEvent = null;
        _messagePublisherMock.Setup(p => p.Publish(It.IsAny<BookCreatedEvent>()))
            .Callback((IDomainEvent e) => sentEvent = e);

        var request = new CreateBookRequest() { ... };
        await _booksService.Create(request);

        sentEvent.Should().Be(...);
    }
}

Then the data access layer with a third party dependency: a micro ORM library.

public class BooksRepository: IBooksRepository
{
    private readonly IDbQueryContext _queryContext;

    public BooksRepository(IDbQueryContext queryContext) => _queryContext = queryContext;

    public async Task<Id> Create(Book book) 
    {
        var command = 
            @"INSERT INTO public.books (name, date) 
            VALUES (@name, @date)
            RETURNING id;";

        var id = await _queryContext.ExecuteAsync<int>(command, new { name = book.name, date = book.date });

        return new Id(id);
    }
}

public class BooksRepositoryTest
{
    private readonly Mock<IDbQueryContext> _queryContextMock = new();
    private readonly BooksRepository _booksRepository;

    public BooksRepositoryTest() => _booksRepository = new(_queryContextMock.Object);

    [Fact]
    public async Task Should_perform_database_action_to_save_book()
    {
        var book = new Book() { Name = "some name"; Date = "2012-12-31" };

        _queryContextMock.Setup(ctx => ctx.ExecuteAsync(
                It.Is<string>(s => s == @"INSERT INTO public.books (name, date) VALUES (@name, @date)   RETURNING id;", 
                It.Is<object>(obj => new { name = "some name", date = "2012-12-31" }))))
            .ReturnsAsync(1);

        var id = await _booksRepository.Create(book);

        id.Should.Be(new Id(1));
    }
}

To end the feature implementation, we have an integration with a third party that wants to receive a notification whenever a book is created.

public class LibraryVZWIntegrationSender : IMessageHandler<BookCreatedEvent>
{
    private readonly IMyMessageQueueSender _msgQueue;

    public LibraryVZWIntegrationSender(IMyMessageQueueSender msgQueue) => _msgQueue = msgQueue;

    public async Task Handle(BookCreatedEvent e)
    {
        var msg = new MessageQueueMessage() 
        {
            Key = "book-created",
            Body = new BookCreatedMessage { Id = e.Id, Name = e.Name }
        };

        await _msgQueue.Enqueue(msg);
    }
}

public class LibraryVZWIntegrationSenderTests
{
    private readonly Mock<IMyMessageQueueSender> _msgQueueMock = new();
    private readonly LibraryVZWIntegrationSender _sender;

    public LibraryVZWIntegrationSenderTests() => _sender = new(_msgQueueMock.Object);

    [Fact]
    public async Task Book_created_message_should_be_enqueued()
    {
        MessageQueueMessage actualMessage = null;
        _msgQueueMock.Setup(q => q.Enqueue(It.IsAny<MessageQueueMessage>()))
            .Callback((MessageQueueMessage msg) => actualMessage = msg);

        var e = new BookCreatedEvent() { ... };
        await _sender.Handle(e);

        actualMessage.Should().Be(...);
    }
}

Issues

What are we testing in the controller, database, and integration sender tests? Are we testing business logic? The only things we’re testing are mapping and parameter throughput.

What benefit do these tests provide? We are confident the throughput works as expected, sure. But what if, for example, something needs to be fixed when querying the database (the schema is incorrect, or we wrote the wrong query)? Or what if our message bus, which the integration sender sends its message to, is broken at runtime (even though our unit tests pass)?

Additionally, we must think about why and when these tests will have to change. Minor alteration to the query? The database test breaks. Any changes to the method signatures of the BooksService or BooksRepository? The controller tests or service tests break.

Solution

As most of the code is close to the boundaries of our application, it would be advisable to remove most of the tests and write a single integration test.

public class BookTests : IntegrationTest
{
    private readonly _apiClient;
    private readonly _msgQueueStub;

    public BookTests(TestFixture fixture)
    {
        _apiClient = fixture.CreateClient();
        _msgQueueStub = fixture.GetMyMessageQueueStub();
    }

    [Fact]
    public async Should_be_able_to_create_book_and_notify_LibraryVZW()
    {
        var request = new BookRequest() { ... };

        var result = await _apiClient.PostAsync("api/books", request);
        await _msgQueueStub.ProcessedEvents(1);

        result.Data.Should().Be(1);

        var message = await _msgQueueStub.GetEvent<MessageQueueMessage>(e => e.Key == "book-created");

        message.Body.Should().Be(...);
    }
}

This isn’t to say that this integration test is foolproof. It might also be brittle due to the database connection, external dependencies, etc. However, we can maintain our production code with one integration test and a unit test for the business logic in our service rather than with four unit tests that hardly verify anything.

Other posts in this series: