Predicting Crowd Testing Efforts

Publish date:

Most of our clients are keen to use crowd testing to increase  the quality and reliability of application, one of the challenge they face is how to budget for this effort.  Crowd is used for different purposes a.      Crowd is used only for additional test execution, effort can be estimated using regular estimation […]

Most of our clients are keen to use crowd testing to increase  the quality and reliability of application, one of the challenge they face is how to budget for this effort.  Crowd is used for different purposes

a.      Crowd is used only for additional test execution, effort can be estimated using regular estimation models

b.      Fixed number of crowd testers are used for a specific period, cost can be estimated upfront

The challenge is  when you want to use  large and wide crowd platform for doing testing, it may be difficult to budget this  efforts upfront. One of the model used is ‘Price per Defect’  while using a large crowd, as the testers are paid based on the outcome not based on the effort,  they are paid based on valid and complexity of defects they find.

How this effort can be budgeted upfront?

One way is to use defect prediction model to predict the number defects,  there are several defect prediction models which requires minimum input to predict,  they can predict for each phase of testing.

You can find more details from  this white papers

http://www.ijcsi.org/papers/IJCSI-9-5-2-288-296.pdf

http://www.sei.cmu.edu/library/assets/defect-prediction-techniques.pdf

Use a defect prediction tool and predict the numbers of defects you will find,  based on that you will be able to estimate cost for crowd testing effort. We used a simple defect prediction tool, which was internally built using T-92-52 and Rayleigh defect prediction model principles, which predicts with very little margin of error

Related Posts

devops

Site reliability engineering

Genesis Robinson
Date icon August 7, 2020

Due to the current state of how we monitor, alert, and log our digital ecosystem, it takes...

testing

Building a culture of quality transformation

Deepika Mamnani
Date icon December 20, 2019

Transforming from traditional testing organizations to quality engineering organizations with...

data

Zombies, wizards, werewolves, and a test automation silver bullet

Grant Volker
Date icon November 21, 2019

Expectations of technology have dramatically changed over the years, creating a demand for...