I was recently involved in a test process assessment for an organization that has implemented an enterprise level agile transformation for their digital initiatives. Here, are my top nine learning’s to keep in keep in mind while developing your digital platform testing strategy for your enterprise wide agile transformation.
1. Building an end to end digital platform typically encompasses 20-25 individual scrum teams focused on developing specific features. This significantly increases the complexity of creating an effective test organization structure wth right set up of lean methods , tools and techniques to meet speed to market needs for the integrated platform.
- a. Create a hybrid test team, comprising of a lean centralized test team at value stream level for the digital platform, and individual testers embedded in the scrum teams for feature testing.
- b. The Centralized team will primarily focus on: system integration, non-functional, regulatory, compliance and usability testing.
- c. The individual feature testing teams, embedded as part of individual scrum teams, will primarily focus on: validating architecture, test design, services testing, security testing, performance testing and system acceptance testing at the feature level.
2. Create a distributed governed user experience testing team. Governance is a centralized function that provides usability guidelines in the form of wireframes, style sheets, mock ups, prototypes, and brand control guidance. The individual validation activities occur at the feature team level.
3. Design verification is a must for performance .This involves best and worst case analysis of performance tolerances and failure mode effect analysis at the feature level and the integrated platform.
4. Use a capacity based estimation model, as opposed to story based estimation, to drive predictability in individual feature testing teams. Target a developer to tester ratio of 1:2 or 1:3; due in part to validation needs from multiple geographies, platforms, locales and devices.
5. Mandate standardized behavior driven “what if then format” to build user stories and map test cases to acceptance criteria across all feature teams. This helps in building testable acceptance criteria for automation and creating an enterprise wide automation framework.
6. Make integration testing your very first priority. Adopt techniques such as Dependency Structure Matrix, Domain modeling and Model based testing. Upon conducting a root cause analysis, you will likely find that 80% of post system integration testing defects are due to integration issues.
7. Test environments set up and test data creation/management will be your biggest bottlenecks. In this environment, we had 20 feature teams (which is typical) with a minimum of four environments per team. Utilizing 100% permanent environments is costly and likely not feasible; to resolve this challenge, leverage 60% service virtualization and a minimum of 50% of test data automation.
8. Leverage crowd sourcing for the first few releases for features that need to be validated in multiple devices and multiple combinations for a diverse set of end-users
9. Continuous test automation is a must. Typical guidelines are 100% test driven development, 80% behavior driven development, 70% regression automation and 50% services automation.
Digital transformation is a business, technology and architectural change. Its success lies in maintaining and exceeding high degree of quality on all touch points. Focus on increasing testing depth first and then breadth.