Unit testing: the hidden part of the test automation iceberg

Publish date:

Unit tests verify the smallest pieces of software in isolation by simulating dependencies. This makes them the fastest, most independent and stable automated tests.

Unit testing: the hidden part of the test automation iceberg

These days it is almost impossible to imagine a test automation strategy without Mike Cohn’s test automation pyramid. The pyramid represents optimal test automation distribution, and unit tests should always form its base. Unfortunately, the test automation community focuses on tools and approaches to automate UI tests rather than unit tests. As a result, the pyramid comes to resemble an iceberg, with only the tip consisting of UI tests readily apparent. It is about time to bring the iceberg above the surface.

Why focus more on unit testing*?

The test automation pyramid is based on the speed and stability of different tests. Automated UI tests represent the small tip because they are dependent on several factors —  data, browser behavior, connection with a cloud-based platform (like Saucelabs)—which make them slow, brittle, and time-consuming to maintain.

Unit tests verify the smallest pieces of software in isolation by simulating dependencies. This makes them the fastest, most independent and stable automated tests. Unit tests provide immediate feedback on the code, so when something breaks down it is easier to debug. An essential asset in a DevOps environment where rapid feedback is the key. Better still, coding becomes more fun for the developer! It is safer to refactor, and the developer can write code faster while making better development choices.

Why under the surface?

Often I experienced that testing is not seen as part of test automation. It’s part of programming and the responsibility lies with the developers. When application code is written, unit tests are often an afterthought.  I’ve met many developers struggling to come up with relevant test scenarios and write the tests. And actually that is not very surprising, since there quite some pitfalls in unit testing:

– Executing the code without actually testing it.
– Testing the result of simulated dependencies instead of your code.
– Testing third party libraries (like JVM getters/setters, Google API)  instead of your code.
– Missing test scenarios due to complex loops, if/else statements and exception handling.
– Asserting too much in one test which results in difficult debug situations.

Yet testers are hesitant to collaborate on unit testing, but who are they to blame? It is obviously the developers’ job, the tests are hidden in the application code and just way too technical. Besides, if the code coverage is 80%, test coverage is also 80%. Right? Wrong. The only thing 80% code coverage guarantees is that 20% has not been tested and that the code has been executed by unit tests, not actually tested. Code coverage can therefore easily create a false sense of security.

How to break the ice?

Get your unit tests above the surface. Mutation testing is a way to determine the actual quality of your tests. With simple plugins (like PITest for Java or Stryker for JavaScript), little effort is needed to run it against your application code. Your application code is mutated to see if your unit tests fail. Should they pass, the mutation survived and the issue lies with the quality of your test set. In a mutation test report, it is easy to identify the mutation test coverage and to see which mutations survived. This provides actionable input for improvements.

And like always, collaboration is an essential success factor. Collaboration between developers and testers will improve the quality of unit tests. The developer guides the tester through the code and the tester helps the developer creating test scenarios. Meanwhile, the tester learns more about code and the developer learns more about testing: a win-win-win situation, if you’d ask me. As we all know, an iceberg under the surface can be quite dangerous, so collaborate to get it above sea-level!

That being said, I am very curious about your thoughts about unit testing. Would you be interested in learning more about unit testing to get immediate feedback on and improve the quality of your application and collaboration between developer and tester?

*Keynote: API testing; another important part of test automation also lies hidden beneath the surface, but for now I focus on unit testing.

 

 

 

 

Related Posts

Cybersecurity

IAM DevOps in Capgemini

Terence Stamp
Date icon August 10, 2018

The Agile methodology has been gaining traction in its adoption throughout the business world. ...

cloud automation

Enterprise DevOps and agility: The secrets to a successful cloud strategy

Gunnar Menzel
Date icon August 8, 2018

Whether you’ve just begun thinking about the cloud or have a tried and tested strategy,...

cloud automation

Reshape the workforce: reskill, redirect, redeploy

Charlie Li
Date icon August 2, 2018

What’s stopping companies from pursuing automation? The answer is DevOps, or rather, a fear...

cookies.

By continuing to navigate on this website, you accept the use of cookies.

For more information and to change the setting of cookies on your computer, please read our Privacy Policy.

Close

Close cookie information