The number of mobile initiatives and mobility solutions we consult on are increasing every week and the upwards trend will continue. Given that the timing of mobile revolution has unfortunately coincided with the global economic downturn our client CIOs and CTOs are under unprecedented pressure to reduce costs wherever possible and deliver with smaller budgets.
Simply put our clients are faced with the conflict to “do something in mobile” and yet “spend the same or less than last year”.
Evidence of this conflict can be seen in the just-released “World Quality report 2012” report which is a global survey of over a thousand companies analyzing the state of their application and quality and testing practices. Over half (55%) of respondents to this report saw their testing budgets frozen or reduced from last year. Optimistically the respondents thought that testing budgets would rise by 2015.
The third section in the report is the more relevant to me as it provides a snapshot of the mobile testing sentiment and draws a somewhat downbeat summary, aptly entitled “Mobile Testing: behind the curve”.
Notable from the survey is that 18% of responding companies say that they do not have enough time to test mobile apps, and 65% do not have the right tools. The 52% who cite lack of devices as a reason not to do mobile testing seems like a red-herring and I wish I were able to drill down deeper into this point: would a company CxO put at risk the usable release of a customer-facing app due to not being able to source some £300 devices to test on? The six main survey reasons for not testing mobile apps are shown below.
Rather surprising from the survey is that the highest percentages of mobile apps that are being tested are in the manufacturing, energy and utilities, and health and life sciences sectors. From the results these sectors seem more likely to formally test mobile applications than the consumer or banking sectors (which made me glance suspiciously at the banking app sitting on my phone’s home-screen).
According to the survey the lowest percentage of mobile testing happens in the public sector with only 27% of organisations in this sector testing mobile apps (see figure below). The next time you are frustrated at some administrative error in dealing with your local council you might be better blaming the lack of testing of the employee’s workforce mobile app than human-error or the back-office systems.
Mobile testing is an area we have much experience of at Capgemini. We have delivered complex projects and highly secure applications that have required deep mobile testing – for any critical line-of-business or customer-facing applications we know that mobile testing is a must. But faced with the pressures to reduce costs and deliver more quickly we find that that some companies will see a separate and long phase of mobile testing as a “cost-too-far”.
So how to square the circle? Mobile design and development fits nicely with iterative development. Looping around the cycle of requirements, design, build and test in smaller chucks means that testing need not simply be a cost “tacked-on” to the end of a project. There are some types of such as penetration and security testing that should only be done towards the end of the project, but for functionality, performance and ease-of-use testing we recommend that this be done iteratively during the mobile project.
We suggest that small trails, POCs and closed real-user-groups are a way that testing can happen in a more cost-effective (and faster) way. The client gets a better-tested app, in smaller usable releases, and the users are engaged earlier and more deeply.
An interesting aspect that is covered in the report is the prioritization of aspects of mobile testing. Interestingly ease-of-use is only half-way down the list of priorities and I would have expected it to be higher: in our experience users will be better served with a smaller set of highly-usable features (which can be built-upon over time) than a larger set of hard-to-use features. See figure below for the priorities.
From the same figure it is also sad (and scary for all of us relying increasingly on financial and highly-personal apps) to see security testing so low with only 18% of responding companies prioritizing this aspect. From our engagements so far this year it has been the highest priority: a large proportion of mobile strategy work within the Capgemini Mobile Apps team is around mobile security and the customers we deal with are rightly concerned of lost and stolen devices revealing citizen, patient, personal details or otherwise confidential information and the possibility of resultant lurid headlines in the press.
Overall an interesting and easy-to-skim report. I would suggest that if you’re interested in mobile testing you read the full report here: http://www.capgemini.com/insights-and-resources/by-publication/world-quality-report-2012–2013/.
You can always email me at firstname.lastname@example.org