The eighth edition of the World Quality Report states that only 1% of the 1600 enterprises with more than 1,000 employees surveyed do not adopt agile development methods. The other 99% use either one or a mix of many methods in different projects. In the same report, we also obtained statistics about the challenges in applying testing to agile development.

Only 1% of these 99% don’t have real difficulties with testing in agile, so 99% of 99% do. That’s 1568 companies in this sample alone. That’s an impressive number. What struck me most was that while Agile development might be the new kid in town that everybody wants to hang out with, this kid still comes with old habits that seem to die hard.

In my experience, early testing team involvement within the software development lifecycle has posed a problem in Prince2 projects; as such, it remains scarce in these projects. It seems that agile development hasn’t altered this process one bit. I also find peculiar the implication of a separate testing team. Testing ought to be an element that is automatically involved in inception or sprint planning.

Independence Incorrectly Perceived and Applied

It’s no wonder that there is a lack of professional test expertise on these teams: if you separate testers and put them on a test team, you’ll never get that expertise to the agile team. It’s also no surprise that there’s a lack of testing approaches that fit in well with the agile development method. If a testing approach is based on the results of a separate test team without taking into account the specific testing in the system team within SAFe®, you will probably never find a satisfactory approach fit for your organization’s agile. 

Separate testing is an old habit, based on (outdated?) principles of validation and verification completed independently of development. What makes me say this? I believe the principle is interpreted in such a way that it creates unnecessary challenges.

The text in the ISTQB syllabus states:

The mindset to be used while testing and reviewing is different from that used while developing software. With the right mindset developers are able to test their own code, but separation of this responsibility to a tester is typically done to help focus effort and provide additional benefits, such as an independent view by trained and professional testing resources. Independent testing may be carried out at any level of testing.

A certain degree of independence (avoiding the author bias) often makes the tester more effective at finding defects and failures. Independence is not, however, a replacement for familiarity, and developers can efficiently find many defects in their own code. Several levels of independence can be defined as shown here from low to high:

  • Tests designed by the person(s) who wrote the software under test (low level independence);
  • Tests designed by another person(s) (e.g. from the development team);
  • Tests designed by a person(s) from a different organizational group (e.g. an independent test team) or test specialists (e.g. usability or performance test specialists);
  • Tests designed by a person(s) from a different organization or company (i.e. outsourcing or certification by an external body).

A lot of testers have memorized this text while studying for their ISTQB-F certificate, and although the text is not wrong, its implementation is. The text does not state ‘Thou shall test in a separate test team’. It explains that testing can be more effective when the tester has a certain degree of independence, and there are several levels of this independence. This can be a different organizational group or organization entirely, but it can also be a person other than the one who wrote the system or somebody else from the team. 

Correct me if I’m wrong, but independent testing must be possible regardless of whether or not you have a tester as part of the agile team. This is because even testing designed by the same person who wrote the software might be described as “low level independence.”

Kill Your Darling

I think at least three of the challenges shown in the figure can be tackled by eliminating separate test teams and welcoming testers into the agile teams.  It will add test expertise to the team, thus ensuring involvement of testing expertise in both inception phases and sprint planning. Because of this early involvement, identifying the right areas for improvement will become less of a challenge as will the lack of an effective test approach. A well-trained tester will also be able to design reusable tests and automate these tests at the appropriate levels. Six challenges solved by killing a darling…not bad at all.

For an in-depth look at the key trends in Testing and QA, download the World Quality Report 2016