Tech Crunch have decided that we need to kill Big Data mainly because the hype is getting silly. Over the last few years every startup and established software vendor threw a ‘social’ sticker onto their products and as Facebook bombed and others have struggled to justify inflated expectations (except for LinkedIn with its actual old school business model of selling stuff that people want to buy). This year the banner appears to be being passed to Big Data as enterprises look to invest in these new technologies so startups, and traditional vendors, all start claiming to be ‘the’ answer to Big Data. As ever with hype there is truth and reality under the hype and that truth is that businesses are looking to ask new questions or use new information sources to make better decisions. Its these questions and sources that are the real new value, the mass of Big Data technologies are simply enablers, often effective enablers but still just enablers.
As ever with technology and technology analyst driven hype there is a massive over-focus on the technology enabler and very little on the actual problems trying to be solved. The normal line in this goes something like ‘Technology X makes it so easy to use you don’t need to be an expert‘, but an expert in what? Sure it might mean you don’t need to understand Map Reduce but are they seriously claiming that a supply chain planner doesn’t need to be an expert in the complexities of a supply chain? That a chaotic system will be reduced to simplicity thanks to Technology X?
What this hype misses, and why I think killing Big Data is too much, is that the real opportunity here is in both those new questions and complexities. Its by understanding those elements that we learn how to leverage the technology as an enabler not by starting with the technology and pretending its the answer to everything.
The Big Data and Analytics challenge is real, both in terms of its complexity as well as in terms of its benefits. What we are seeing right now however is a repetition of the hype that was applied to Social Media, BPM, SOA, CEP, EAI, ERP etc etc etc and it comes down to the same thing,
There is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity.
So when looking at the Big Data hype remember that these new technologies are enablers but it requires other changes as well in terms of how you govern data, model information, model analytics, make decisions and the whole culture of decision making within your organisation to actually deliver the impact that you need. As with other technologies what we will see is the technology lose its luster and organisations slowly make the changes required to leverage their hyped investment. A better strategy is to look at the broader changes you need to make, and critically the questions you want to answer and complexities you want to over come and only then look at which technologies will be the right enabler.
Solving the complexity problem of Big Data is not about the technologies, its really about the culture, just like MDM did when it was the hyped technology of the day.