Unit testing: the hidden part of the test automation iceberg
These days it is almost impossible to imagine a test automation strategy without Mike Cohn’s test automation pyramid. The pyramid represents optimal test automation distribution, and unit tests should always form its base. Unfortunately, the test automation community focuses on tools and approaches to automate UI tests rather than unit tests. As a result, the pyramid comes to resemble an iceberg, with only the tip consisting of UI tests readily apparent. It is about time to bring the iceberg above the surface.
Why focus more on unit testing*?
The test automation pyramid is based on the speed and stability of different tests. Automated UI tests represent the small tip because they are dependent on several factors — data, browser behavior, connection with a cloud-based platform (like Saucelabs)—which make them slow, brittle, and time-consuming to maintain.
Unit tests verify the smallest pieces of software in isolation by simulating dependencies. This makes them the fastest, most independent and stable automated tests. Unit tests provide immediate feedback on the code, so when something breaks down it is easier to debug. An essential asset in a DevOps environment where rapid feedback is the key. Better still, coding becomes more fun for the developer! It is safer to refactor, and the developer can write code faster while making better development choices.
Why under the surface?
Often I experienced that testing is not seen as part of test automation. It’s part of programming and the responsibility lies with the developers. When application code is written, unit tests are often an afterthought. I’ve met many developers struggling to come up with relevant test scenarios and write the tests. And actually that is not very surprising, since there quite some pitfalls in unit testing:
– Executing the code without actually testing it.
– Testing the result of simulated dependencies instead of your code.
– Testing third party libraries (like JVM getters/setters, Google API) instead of your code.
– Missing test scenarios due to complex loops, if/else statements and exception handling.
– Asserting too much in one test which results in difficult debug situations.
Yet testers are hesitant to collaborate on unit testing, but who are they to blame? It is obviously the developers’ job, the tests are hidden in the application code and just way too technical. Besides, if the code coverage is 80%, test coverage is also 80%. Right? Wrong. The only thing 80% code coverage guarantees is that 20% has not been tested and that the code has been executed by unit tests, not actually tested. Code coverage can therefore easily create a false sense of security.
How to break the ice?
And like always, collaboration is an essential success factor. Collaboration between developers and testers will improve the quality of unit tests. The developer guides the tester through the code and the tester helps the developer creating test scenarios. Meanwhile, the tester learns more about code and the developer learns more about testing: a win-win-win situation, if you’d ask me. As we all know, an iceberg under the surface can be quite dangerous, so collaborate to get it above sea-level!
That being said, I am very curious about your thoughts about unit testing. Would you be interested in learning more about unit testing to get immediate feedback on and improve the quality of your application and collaboration between developer and tester?
*Keynote: API testing; another important part of test automation also lies hidden beneath the surface, but for now I focus on unit testing.