We have seen a significant shift in the IT industry over the past decade, both in terms of technological advances (AI, the blockchain, etc.) and in process models and frameworks (agile, CI/CD, etc.). I see this reflected in QA budget trends and in the way we are doing testing/QA.
During this timeframe, QA spend as a percentage of IT budget ranged from 18% to 35%. The good news is that, although the predicted levels initially went up to 40% by 2018, we have remained steady at 26% in 2017 and 2018.
The spike in QA share of IT budget during 2016 (31%) and 2015 (35%) can be explained by the following:
- Increased adoption of agile, DevOps and cloud
- Including test data management (TDM) and test environment management (TEM) under the QA budget, instead of it being under production support or infrastructure budget
- Last but not the least, the increased focus on customer experience and security aspects at that time, thanks to the increased mobile device access.
All these triggered a wave of investments in infrastructure and reskilling initiatives. Since then, realizing targeted benefits and efficiency gains from maturity, brought the QA budget back to 2014 levels (26%).
When considering the QA budget in the context of efficiency and cost containment, three key factors must be noted:
- QA efforts: There is a significant increase in the number of release cycles and test cycles. But, thanks to automation, analytics across SDLC, and virtualization of test data and test environments, the proportionate cost and effort have fallen. This is reflected by the drop in spending on human resource aspects of QA, which has steadily dropped from 35% in 2014 to 26% in 2018.
- Efficiency play: On the system-of-records side, we are seeing a lot of efficiencies being realized from investments. However, on the system-of-engagement side, huge investments are driving costs. They both exist side by side and negate any efficiency gains.
- Distributed model: As for measuring and reporting of QA efforts, especially in the context of agile, DevOps and prevalence of SDET roles make it less accurate because everyone is doing testing, not just specialist testers.
Increased spending on the hardware and infrastructure side of QA spend is a surprising trend that can be attributed to organizations testing the waters in the artificial intelligence, IT and robotic process automation space and investing in setting up sandbox environments.
Overall, intense investment was followed by period benefits realization from increased efficiency. With agile and DevOps now becoming mainstream, we will likely see increased investment in lifecycle automation, AI and RPA in testing in the near future. As with every technology hype, we can expect a slight increase in QA budgets in the short term before efficiency gains.
To achieve increased efficiency and contain costs, define elaborate and consistent ways to measure, report, and track testing efforts in agile and DevOps. Without this, it will be impossible to measure and report the costs contained or the improvements in efficiency.
Download the tenth edition of the World Quality Report 2018 to get deeper insights into the latest trends in the world of testing.