As I've stated before, during the last decades companies have become data hoarders. In order to extract additional insights yielding competitive advantage, they've not restricted themselves to the necessary data for their products and processes, they've started to store additional data as well. IDC Research projects total data storage will soon exceed 1.8 zettabytes. In more concrete terms, over 57 billion 32GB iPads.
This phenomenon is described as "big data". This terminology seems to refer very much to massive database tables and data warehouses. But while structured data is growing rapidly, unstructured data like documents, multimedia and social activity streams is increasing in magnitude even faster. Complicating factor is that the analysis of all of these unstructured data has gotten harder not only because there is more of it but because it comes from new sources. Blogs, Web comments and other information comes in the form of unstructured data, which can't be crunched the way relational databases are. The need to mine different types of content has led to new data analysis platforms. So while big data is being looked upon as a tool for competitive advantage, gauging trends to supporting sciences and determining consumers preferences, it is also a big challenge to manage. It means managing fast increasing:
- velocity : creation, read and delete procedures of data including accessmanagement and governace
- volume : Storage including green IT
- variety: Type of data and documents including digital sustainability.
New tools (Hadoop, NoSQL) are emerging to analyze big data, but organizations still struggle with how they can truly harness massive amounts of data to improve decision-making and performance. On the one hand it means that the market for new tools to manage and exploit big data is has great growth potential. On the other hand it is clear that with such a variety of challenges and new developments, companies need to develop a big data vision before they start exploiting it as an opportunity.
Become aware of its challenges, possibilities and risks.
Deploy a big data strategy.
Make people aware of the consequences on data usage, products and processes.
As always; think big, start small.
But most of all, the implementation of big data 'ain't over till it is over'.
Meaning that you have to fully implement it in the companies structures, layers and vaults to fully exploit the potentials big data can offer.