The software industry has for a long time been dominated by industrial views and beliefs. The universally accepted paradigm was in fact a copy-paste of old industrial routines and theories. Essential in this landscape of knowledge, views and practices is the Taylorist conviction that executable tasks, to be performed by workers, must be prepared, designed and planned by superior staff and must be overlooked by hierarchical supervisors during execution. Quality is assured by allowing good and rejecting bad batches of work results. Monetary rewards are used to stimulate desired behavior or punish unwanted behavior.
Studies and research have regularly documented the serious flaws of this paradigm. Many shortcomings and errors were well beyond reasonable levels of tolerance. The unfortunate response seems to have been to lower the levels of tolerance. It was accepted that only 20-25% of software projects are successful. The criteria of success, i.e. the expectation of time+budget+scope, can be disputed, but it is what the paradigm promised. Quality suffered, and it was accepted that over 50% of the delivered features were never used.
Although it was not widely and consciously admitted, but this did put the software industry in a serious crisis. A crisis that we tried to overcome by fortifying the industrial approach. More plans were created, more phases, more designs. More details were added to the tasks to better direct the workers. The supervision was increased. And still, little improved. Too much flaws, defects and low quality had to be tolerated.
But, although it took some time, inevitably new ideas and insights started forming upon the observation of the -significant- anomalies of the industrial paradigm. The seeds of a new worldview were sown. This new worldview was formally named ‘Agile’ in 2001. A new paradigm for the software industry was born; a paradigm that thrives upon heuristics and creativity, and that restores the respect for the creative nature of the work and the intelligence of the ‘workers’ in the software industry.
By definition a shift to a new paradigm takes time. It seems though that the software industry has good reasons to move fast. The existing flaws are important, widely known and the presence of software in society is ever-growing. But the old paradigm seems to have deep roots. It even keeps on being educated as the most appropriate one. Therefore we have reasons to be very skeptical about a gradual evolution, a slow progression from the old to the new paradigm, from waterfall to Agile.
The chances are quite high that a gradual evolution will never surpass the surface. New names will be installed, new terms and new practices, but the thinking and the behavior of people and organizations remains exactly the same. Even worse, the essential flaw will remain untouched, the disrespect for people, treating people as mindless workers. Only the organization will be touched, the management practices, and the top-down disrespect will remain intact.
Too much graduality will leave existing data, metrics and standards in place, and the new paradigm will be measured against these old standards. But different paradigms consist by nature intrinsically of fundamentally different concepts and ideas, often even mutually exclusive. In general, no meaningful comparison is possible. Now, here we have another strange phenomenon in the software industry, giving additional evidence of the deep grounding of the old ways. In 2011, the Standish Group included for the first time Agile software projects in its yearly Chaos research and report. It showed that, even measured against the old expectations of time+budget+scope, Agile was substantially more successful. So, on the one hand Agile is becoming important enough for the Standish group to collect data upon, data that even show more success. On the other hand, what is blocking faster adoption of Agile? Are we replacing the belief in the old paradigm by the belief that a very slow, very gradual shift is better, ignoring that this factually is a status-quo situation?
In the 11 years that have passed since the formal becoming of Agile, Scrum became the leading Agile framework. Scrum has proven to be a great help in installing the new paradigm by making it tangible and comprehensible. Scrum offers a positive alternative to the ancient control models that don't work anymore.
The house of Scrum offers an open view on the world, while protecting from rigid behavior. Its inhabitants remain flexible at all levels to better deal with uncertainty, and shape -not predict- the future. They sense, probe and adapt at all levels. Scrum drives building better software products, and faster. In 30 days, or less. But, most of all, it restores energy and work pleasure for all involved.
The distinct rules of Scrum help in getting a grip on the new paradigm. The small set of rules allows better absorption of the new ways. That is still a giant, first step to be made. But as we grow in incorporating the new behavior we will probably gradually let go of the formal demands and just practice it. We will have entered a new state of being, the state of Agility, a state of constant change, evolution and improvement.
The time that it takes to make the initial shift depends on determination and hard work. It’s a choice, not a must. It’s one way to improve the software industry. Feel free to join.