You Are Writing Too Many Unit Tests
From Testing Pyramid to Testing Trophy
There is probably an unwritten rule among conference speakers that if a presentation is about testing, it has to start with the testing pyramid. This concept, which was extremely relevant when introduced back in 2004, has shaped the way we approach software testing for two decades.
It split all testing efforts into three layers, each representing a different scope and approach to testing.
At the base of the pyramid are unit tests, which are the most granular and numerous. They focus on testing individual components or functions in isolation, ensuring that each piece behaves as expected. They are crucial for identifying fundamental issues early in the development process.
The middle layer consists of integration tests, which are fewer in number but larger in scope than unit tests. They examine how different units or modules of the application interact with each other, checking for issues in the interfaces and data flow between components. Integration testing helps to catch problems that unit tests can’t, as it focuses on the connections and interactions within the system.
At the top of the pyramid are end-to-end tests, which are the least numerous but the most comprehensive. E2E tests simulate real user scenarios, testing the application as a whole to ensure that all components work together seamlessly in a production-like environment. This level of testing is crucial for verifying the overall user experience and the system’s functionality in real-world conditions.
So the guideline is to write many unit tests, but exactly how many are enough? Code coverage was introduced to answer this, but it led to an unexpected issue. We began chasing high coverage numbers, sometimes neglecting the real goal of testing which is software quality. This approach, focusing on quantity rather than quality, can create a false sense of security. Moreover, aiming for 100% coverage can be unrealistic and may lead to overlooking other critical tests, like integration and end-to-end tests.
Some examples of redundant unit tests that I often come across are:
it('renders the component', () => {
const wrapper = mount(MyComponent);
expect(wrapper.exists()).toBe(true);
});Only checks if the component renders, without verifying any specific behavior or functionality. It’s a basic existence check that contributes to coverage but adds little value in terms of assuring component quality.
it('has a specific class name', () => {
const wrapper = mount(MyComponent);
expect(wrapper.classes()).toContain('specific-class');
});Checks for the presence of a CSS class but doesn’t interact with the component or test how its state might change in response to user actions. While it might increase coverage, it overlooks the dynamic aspects of the component.
it('renders list items', () => {
const wrapper = mount(MyComponent, {
propsData: { items: ['Item 1', 'Item 2'] }
});
expect(wrapper.findAll('li').length).toBe(2);
});While this test checks if list items are rendered, it’s primarily testing that the framework directive
(v-for)works rather than any specific logic within the component. The test increases line coverage but doesn’t validate any custom behavior or logic of the component.
it('starts with a count of 0', () => {
const wrapper = mount(MyComponent);
expect(wrapper.vm.count).toBe(0);
});Confirms the initial state of a data property but doesn’t assess how the state might change or if the component behaves correctly in response to events or user input.
These examples show how focusing solely on increasing coverage can lead to writing tests that don’t necessarily contribute to the overall quality or robustness of the application. Effective unit tests should aim to cover meaningful scenarios, edge cases, and user interactions, rather than just fulfilling coverage metrics.
The whole testing suite should improve code quality, catch bugs, and ensure the software performs well in real-world scenarios.
This is where the Testing Trophy offers a solution. It suggests a balanced mix of different test types, emphasizing the importance of integration and end-to-end tests. The idea is to maintain a healthy number of unit tests, but not at the cost of the bigger picture, ensuring that the system works well as a whole.
At the base of the trophy, we have Static Testing. This includes linting and type-checking. Linting has evolved tremendously over the last few years and it’s now the first line of defense against common problems like syntax errors, type mismatches, and style violations..
The next layer up, but smaller than in the traditional testing pyramid are Unit Tests. They focus on individual functions or components in isolation. They are crucial for ensuring that each part of the application behaves as expected. While important, they are not overly emphasized in the Trophy model.
Then we have a big layer for Integration Tests. They are really important because they check how different parts of your app work together. They find problems that might be missed when you only test parts in isolation. From experience, these are often the tests that prevent breaking production and that's why we give more attention to them than before.
Lastly, we have end-to-end tests. They are super important because they check the entire app from start to finish, just like a real user would use it. They test the whole stack from clicking buttons and filling out forms on the front end to making sure data is handled correctly in the back end. While they can take more time and effort, they ensure that the whole app works as expected.
Additional Resources
The concept of the testing trophy is not mine. It was introduced by Kent C. Dodds. Take a look at the following resources if you want to learn more.
