(1) Introduction—Weekend Testing
Weekend testing is an unconventional way of testing that is influenced by a context-driven software testing philosophy. In a short span of time, it has traveled from Bangalore to the rest of the globe. Weekend testing is about testers signing on to collaboration platforms such as Skype or Google Talk and getting started with their testing at a specified time as announced on the weekend testing website. In the group of testers, a tester assumes the role of facilitator to bring seriousness to the mission for the session. The bugs discovered in the course of testing are logged in a source bug database (like Bugzilla) and followed by a debrief session. The session transcript is then posted with the results for everyone to see and analyze. The results are also shared with the client as per the testing assignment.
(2) Weekend Testing - Advantages and Disadvantages
Weekend testing thrives on the use of open source and free tools. With so many willing, skilled, and available testers, weekend testing has become a popular forum from which to learn, network, and earn money. With a wide variety of products to test and a myriad of testing missions to try out, there is something new to learn from every session. With people across the globe participating in these forums, we gain access to an interesting cross-section of skills and cultural trends. Although we are seeing a gradual adoption of this method, we must be wary of any risks it can bring to supplier reputation, security threats to customer knowledge assets, and other operational challenges.
Weekend testing lends itself well to areas such as localization, language, and mobile app and device testing. Key advantages include:
- More Tester Interactions and Tester Clubs: Each tester builds upon their respective experiences and this collaboration enriches the quality of testing.
- Demonstration of Testing: Questions from “How can you complete testing without test cases?” and “How can you repeat testing?” to “How will client accept that we tested?" are easily answered thanks to a transparency policy. Testing remains open for scrutiny by anyone at any time. One can witness testing happening right in front of his or her eyes.
- Tester Opinion and Better Open Source Software: Weekend testing thrives on the use of open source software for tracking bugs; as a result, clients who are averse to spending need not incur costs for purchasing commercial testing tools.
- Scalability and Time to Market: Midsized software suppliers may be forced to increase the team over limited span of time. Weekend testing can help with a vast pool of skilled and experienced testers. Moreover, as weekend testers spread across the globe, customers have the benefit of locating resources in their respective time zone.
- Localization and Language Testing: Weekend testing is a forum that consists of testers across the globe from various countries, and cultures. Testers are readily available for localization testing and validation. Consequently, procuring a weekend tester for a specific language may be easily achieved.
- Device and App Testing: Mobile applications need to be validated on a broad spectrum of elements that include mobile hardware, network, and OS. Software suppliers are struggling to build mobile labs that host these devices and as such, are using emulators in the absence of devices. Weekend testing solves this logistical problem, as the tester pool is so vast. Furthermore, the testing teams have a broad array of devices and OS on which the applications may be deployed and tested. This eliminates the need to test the applications on emulators. In addition, the applications are released in some regions ahead of others, and testers from these leading regions could provide the required experience for other lagging regions.
While weekend testing is an interesting and evolving phenomenon, there are concerns including the following:
- Inadequate Security: Weekend testing does not undertake adequate measures to secure customer IPs, secure an ODC, sign NDAs etc. practiced by customers dealing with offshore service providers. Testing is usually done in an informal and relaxed environment that could make clients skeptical to share their IP with them.
- Inadequate Alignment with Business: Weekend tester’s forums may not be able to swiftly respond to changes in strategy taken by their customers during the project. Lack of a proper governance model and transition methodology hinders the weekend testing projects from responding to changing customer needs. It may therefore pose a risk to project delivery, particularly for brand new implementations.
- Inability to Adopt DevOps: Weekend testing relies more on open source tools and manual testing and therefore does not align well with DevOps delivery models, which promotes faster and quicker testing and integration of Development Test Operations.
- Governance Issues: Weekend testers often abandon tough testing sessions, leaving testing assignments incomplete for several days and therefore disrupting test continuity. Sometimes, core testers have busy weekend schedules and cannot give adequate time and commitment to their testing assignment. This greatly impacts project timelines. A lack of weekend facilitators or moderators leaves testing directionless and without structure.
(3) References from Capgemini Sogeti HPE World Quality Report
Quality is considered implicit these days. Consequently, concentration is shifting on how project testing costs can be optimized and how time-to-market can be improved as a measure of testing success. This year’s World Quality Report cites that 42% of IT executives want cost optimization from their QA function and 37% of them desire faster time-to-market. Weekend testing can help meet these objectives and lower the cost of testing for vendors (by employing weekend testers) as well as for the customer.
Weekend Testing is a form of crowdsourcing. The World Quality Report of 2016-2017 has studied the use of crowdsourcing by respondents. The primary focus on achieving maximum cost efficiency in the form of operational test factory models and captive TCOEs is moving toward a demand for more flexible, nimble, and easily accessible TCOE models like crowdsourcing. It is interesting to note that amid the shift, organizations still seek managed services models apart from vendors in the setup of test scenarios. While a managed service model provided by an expert QA vendor ranks the highest in importance (49% of respondents), the use of a crowdsourcing solution model for quick test capacity is used by 47% of respondents, alongside a decentralized TCOE for improved agility and efficiency. This puts crowdsourcing ahead of a captive TCOE (45%) and a TCOE in an offsite location (42%).
Important Elements in Setting up test Operations
(4) How Can Large IT Organizations Ride the Weekend Testing Bandwagon?
Although there are few demerits in weekend testing, the power of weekend testers can still be leveraged tremendously. Some IT service providers have formed an alliance with crowd platform providers such as Applause as part of its Liquid Workforce Strategy, featuring more than 250,000 experienced quality assurance testers from around the world. Applause complements in-house testing professionals to provide a range of testing services and geographic coverage to their customers, especially in the areas of mobile devices, desktops, kiosks, smart TVs, wearables, and IOT testing.
A formal governance model can help integrate weekend testing teams with the test engagement teams of IT service providers. Weekend testing teams report to the meeting facilitators who typically consist of Test Leads or Managers of test engagement. Such governance could include monitoring and reviewing of daily project activities, including weekly work schedule, progress made, and any course corrections required to meet the quality, and schedule of the transition. Open issues are highlighted and actions are taken to resolve them. However, work distribution across locations, quality level, and SLA performance need to be verified. Staffing matters such as team ramp-up and co-location could also be discussed through formal governance.
Weekend testing is an interesting evolution in software testing but its scope is currently limited to projects involving plain vanilla testing wherein traditional models like Waterfall, V are employed. It definitely provides advantages such as a larger and more diverse knowledge base and increased tester experience, which lead to effective testing.
With the latest advances such as DevOps relying heavily on automation tools, scope for weekend testers may be limited. Weekend testing teams lack a defined structure and process and may not be able to win over the confidence of the customer. However, if a large testing organization leverages the weekend testing concept for areas such as IoT and ensures adequate governance and accountability, the concept can be employed with confidence.
1. Capgemini, Sogeti, HPE, World Quality Report #WQR2016
Main Author: Renu Rajani, Vice President, Capgemini, Testing Global Service Line; firstname.lastname@example.org
Contributing Author - Rohit Sharan, Solutioning Manager, Capgemini, Testing Global Service Line, email@example.com
For an in-depth look at the key trends in Testing and QA, download the World Quality Report 2016 http://ow.ly/EUgr304DKle