Firstly it is worth noting that Big data is really two things: the massive new volumes of data becoming available from the connected world and secondly the new technologies that have emerged to support these data sets (eg Hadoop). These combine to radically change the way we look at Business Intelligence. Here is how:
1. The shift to: “Store everything and distil on demand”. We call this the Business Data Lake. It is a fundamentally new way of looking at how we manage data that the new technologies have enabled. It satisfies both the very diverse needs for information that organizations have today and the need for rapid turnaround for new information requests. It is well explained in a couple of videos: Explaining the Business Data Lake & The Opportunity for the BDL.
2. Exploiting new big data sets. When we started doing big data we thought it would be relevant for Consumer Products and Financial Services companies and the rest wouldn’t need it. That has changed. There is not an industry sector that we are not working with on big data. New data sets are popping up from the IoT and supply chains and assets and business transactions. It is evident that those companies that do not start to tap into this will be left behind.
3. Advanced & Predictive Analytics – the volumes of data and analysis that needs to be carried out are just too much for individuals to sift through with traditional reporting and slice & dice tools. Organizations are looking for solutions that provide guidance and advice to decision makers.
4. Data Optimization – As new technology (Hadoop, inMemory, MPP) becomes mainstream and affordable, so companies will move from monolithic data warehouses to re-organize their data with striped storage –eg Hadoop for volume, RBDMS for core and inMemory for fast access. Explained in: Data Optimization
5. Insight at the Point of Action (process Integration) – high performance technologies (eg inMemory) are allowing complex analytics to be performed at speeds that are compatible with business transactions. This means the results of the analytics can be built into business transactions to automate complex decision making.
In most cases this will be an evolution: new platforms, technologies and data sets will spring up alongside the existing EDW solutions but once organizations begin to see the value of the “new world” – BI will change forever.