Testing Philosophy – I code therefore I test | Opinionated Pattern Picking

Welcome to Opinionated Pattern Picking (OPP), a monthly session we run at Arinco to foster an environment of discussion and learning for our application developers and anyone else interested. Each month the team discusses a topic and attempts to elect a “best default” pattern for developers to use on future projects. View the rest of the blog series here.


What solution are we designing for?
Instead of looking to build an enterprise solution, we take the approach of starting with a straightforward solution: we create a solution that involves an API and some data that can be retrieved either in response to an end-user request or by a daemon service running in the background. This simple approach is still quite detailed, providing us with enough substance to delve into before adding further complexity.



November – Unit Tests vs Integration Tests vs End to End Tests

The opinionated pattern picking session this month focused on the philosophy of unit testing, integration testing, and end-to-end testing. We deliberately avoided how to implement tests in favour of discussing when and where different types of tests should be used.

Pattern Picking

Unit Tests

The group began by exploring unit tests, which was described as a contract at a point in time that confirms code or functionality works. To begin we defined, our thoughts on what exactly a unit test was:

  • Test individual units of code in isolation
  • What is a unit?
    • Business unit – Logical sequence of functions
    • Functional unit – E.G String Manipulation function
  • In isolation of integrated services – Mock of data remove of external systems
  • A testing of business rules not external processes
  • Lean and quick, so it runs quickly in pipeline

This sparked discussion about unit tests being more than just method testing – they could represent a business unit or flow. Several participants emphasised that unit tests work with code in isolation, often using mocked dependencies. Participants highlighted that unit tests are particularly valuable for pure logic and calculations, while others emphasised their importance for code coverage and testing different paths through the code.

A significant point raised was that every line of test code written is a liability, as it needs to be maintained and cared for. This led to discussion about balancing test coverage with maintainability, noting that more tests don’t necessarily mean better code.


Integration Tests

The discussion then shifted to integration tests, which the group identified as involving actual integration to services like databases and storage accounts. There was substantial discussion about mocking boundaries, particularly around services like Event Grid versus Event Hub where local emulation capabilities differ. This sparked debate about what should be mocked versus what should use real integrations. The group acknowledged that integration tests tend to be heavier to run than unit tests. Some organisations split integration tests into two categories: quick ones needed for deployment and delayed ones for additional coverage. Several participants noted that having too many integration tests might indicate a micro-service is doing too much. The group discussed how UI testing fits into the picture, noting that integration tests can be particularly valuable for testing UI functionality and complete flows, rather than just focusing on backend API tests. When testing UIs, they suggested that testing the whole flow through integration tests might be more valuable than writing separate tests for backend APIs.


End to End

When discussing end-to-end testing, the group characterised it as the most expensive in terms of effort. There was general agreement that end-to-end tests are often best handled by testing teams rather than developers, as they mimic what real users do on the system and test the complete flow from UI through to database and back.


Testing for accountability

Tests were presented as a contract representing a specific point in time where code or functionality works and meets business requirements. This contract serves two purposes:

  • Helping future developers understand how code works and what happens when things go wrong
  • Helping reviewers verify code functionality without having to deeply understand implementation details. When tests break, it’s viewed positively because it means either requirements have changed or functionality has evolved, making the change explicit through the broken contract.

Opinionated Pattern Picked

The group ultimately converged on “lean integration tests” as a preferred approach for web applications. This involves using dockerized databases where appropriate. Writing primarily unit tests around functionality and focusing on testing critical paths. Additional work was done to further define what a unit is, which got refine further to:

What is our definition of a unit:

  • Unit: A unit of work
  • Work: A transaction boundary of functionality

 They emphasised using actual integration points rather than mocks where possible, while keeping the number of end-to-end tests minimal. The group pointed out that while this approach works well for web apps, different types of applications like graphics engines or scientific software might need different testing strategies.

Rather than aiming for specific code coverage percentages, the group emphasised the importance of testing critical paths and ensuring tests would break when underlying functionality changes. Another less technical consideration for the success of test was the role of tests in code review and communication. As tests serve as documentation and proof to code reviewers that acceptance criteria have been met, helping demonstrate understanding of requirements. This was seen as particularly valuable when reviewers weren’t involved in the implementation details.

Read more recent blogs

Get started on the right path to cloud success today. Our Crew are standing by to answer your questions and get you up and running.