Mastering continuous testing

Moving towards continuous delivery is an ambitious journey. Much like a tall building, it cannot be built without solid foundations. To master quality in this context, we must transform our way of working and adapt to our organizations. Significantly, people must be at the heart of this transformation. Here are some key elements to consider if we want to reach the holy grail of controlled quality in the world of continuous everything.

Automation

At risk of stating the obvious, let’s start with the pillar of continuous testing.

“Should I automate?” That is no longer the question. Continuous deployment cannot happen without continuous testing and continuous testing cannot take place without automation. But what to automate? Efficiency and ROI impose that we approach this question with caution.

Who should automate? Since teams have to be very autonomous, it is reasonable to make functional people automate functional and user tests by themselves. There are interesting scriptless tools that allow this and, in this case, there is no need for heavy knowledge transfer from functional people to test “automaters.” But experience also shows that it’s not that easy for them to think like testers, to consider best practices, criticality, strategy. They need to be coached and guided in their new practice. And don’t forget that their real added value is in functional and business activities, not in testing.

Automation is not always an accelerator since automation takes time and effort. An automation strategy is needed to determine the priorities and criteria that make it interesting to automate a given test. From unit test to user test, the approach must be defined and based on common sense (recurrence, technical effort, maintenance effort, criticality, etc.) and automation must follow good practices to be efficient and reliable. Modularity, reusability and all the standard rules have to be applied to obtain good results. Continuous everything implies that we always have less time. There is no space for wasting it.

Collaborate

This is an Agile credo that becomes increasingly true when we apply Agile principles to the entire company with SAFe® or similar methods. Collaboration and transparency make it possible.

Agile Dev —> Agile Testing

The story began with Agile development. But personally, I know of only one big project that has applied best practices for Agile testing (the right test pyramid, TDD, BDD, ATDD, etc.). Is it a coincidence if this project is also exceptional in terms of mastering quality?

Thus, if we want to build solid foundations for this transformation, let’s apply the right rules before thinking about the color of the walls. Agile testing is a topic by itself and it must not be neglected.

More coaching and less management

Agile needs autonomous teams. This is also true for Agile at scale. Therefore, competency must be embedded in the teams. More than ever, test managers have to be teachers, much in the same was as testers must be in a scrum to show to developers how to benefit from the power of test and automation.

Data and environments

To run tests continuously, data and environments must be always ready. This is part of the harder objectives to reach. While we can use a mocked environment for unit tests, this is not possible for higher-level tests. At this point, we need to automate the data test set. This can be done by a query (in a production replication, for instance) or by generating these data. Appropriate tools can also be used, but this effort is almost the same as for test automation.

Models

In MBT (model-based testing) we trust. Modelized business processes provide many benefits. A model is visual, thus simpler to understand and to share. This can become a knowledge base used as the only reference (to avoid multiple sources). This can be a basis to define critical paths that will be the nominal scenarios that must be run as often as possible.

Dedicated tools provide automated test scripts from the business processes. They can also compute impact analysis when a change is applied, which is very valuable when designing the regression tests set (when the tool is not able to build it by itself). Models provide a lot of benefits for efficiency, reliability, and maintenance.

AI support

Among the multiple announcements about magical tools emerging from AI, there are also some realistic and proven ones that might help our journey towards continuous testing. For instance, Cognitive QA® is a solution that can use testing data to provide an optimized regression test plan, analytics dashboards about quality, and predictive information for optimizing effort and time.

To conclude, continuous delivery is a lofty ambition. It implies mastering quality in a continuous process. The principles we know for standard life cycles have to be revisited and organizations must be adapted. Automation is an essential component of the necessary transformation, but it must be used methodically if it is to provide results expected. Experience shows that various elements can affect the transformation, starting with the human factor.

Hence, be aware of all the basic elements that must be considered to make your journey easier, and even possible.

Download the first edition of the Continuous Testing Report to get deeper insights into the latest trends in the world of testing.