“Who loves not wine, women and song; remains a fool whole life long”. This is an old German expression which means having fun. So why do I mention this? I am very enthusiastic about the upcoming trend in decision making to rely more on statistical analysis. The umbrella term for this is called analytics. Recent books about this topic are: Competing on analytics by Thomas Davenport or Super Crunchers by Ian Ayres (www.supercrunchers.com). They send the message that you are no longer a nerd if you think that regression is cool. For the non mathematical persons: Regression is a statistical procedure that takes raw data and estimates how various causal factors influence a single variable of interest.
Ayres describes in Super Crunchers how analyzing massive databases can provide greater insights in human behavior. One of his case studies involves the work of Oren Ashenfelter. Oren can assess the quality of Bordeaux wines based upon regression (basic variables are: rainfall & temperature). There is much skepticism about his work among wine connoisseurs. Robert Parker – probably one of the world most influential wine writers – said: “it is really a Neanderthal way of looking at wine”. However it turns about that Ashenfelter predictions are much more reliable than the old fashioned way.
In October there was a global democracy week. Over forty broadcasting companies worldwide paid attention to this on radio and TV. Here in the Netherlands we had a mock competition for a new national anthem. The current one which is called The Williams song – or Wilhelmus – is considered to be the oldest one in the world. You can hear all the competing songs on this website: http://www.wijzijndebaas.nl/page/liederen. As a follow up to this program there was a segment on a company called Platinum Blue. They have come up with a tool that is called Music X-Ray. This is a complex computer program that breaks songs down into 30 or so component parts’ including rhythm, melody, harmony, beat, cadence, timbre, pitch and gives each a number. All hit songs fit the same pattern. If your song matches the pattern this increases the change of success dramatically. They claim that human decision making (does it sound like a hit, can we promote it like a hit) delivers a 10% success rate. However if you add a third factor – statistical analysis – this score goes up to 75%. You can see their presentation on you tube: http://www.youtube.com/watch?v=8lUewWlLgLo. Just the same as with the wine there is a lot of criticism. The X-factor judge Louis Walsh said: “The only technology I would use to pick a hit is my ears. I would not listen to a computer,”
The pattern is the same in both cases. The traditional or old fashioned way of doing business is challenged by a new approach based upon statistical analysis leading to much skepticism. In my view this discussion is too much binary digit, true or false (I wanted to say black-and-white but I am still a technology consultant). In both of the above mentioned cases it is just a tool or technique – a guide to making the final decision, which always comes down to a human being. It is not about replacing the human decision making process with statistical analyses. But on the other hand there is enough evidence to support the fact that adding analytics to the decision making process leads to increased results. This can be done by creating a dialogue between Business Intelligence tools and their users, instead of a one-sided approach of data delivery only, leaving the decision making process to the end user. This going back and forth between user and tool can lead to something that I call intuitive intelligence: leading to insight without (much) thinking.
On a final note. It is great if any BI or analytical system can support you in knowing thing, having new insights, the power to know. But it is much better if you are able to make these insights really actionbable, the power to act. This is where the real added value of information lies and where any intelligent enterprise shoud be going.