Thriving On Data #2 – Intelligence Inside
‘Big’ Data is not really about volume; it is about the ability to analyze and act in real time using data from sensors, transactions or interactions, both from inside as well as outside the organization. Data can be used to solve tougher business problems, create more competitive advantage and make much better, informed decisions in a tightly connected world. Where placing data in the hands of many previously was a separate process, exploration now has become a part of daily work. With this comes the increased need to create insights on the fly, done by business users instead of remote IT experts.
Werner Heisenberg was a German physicist and one of the key creators of quantum mechanics. In 1927 he published his uncertainty principle for which he is best known. This principle states: “It is impossible to determine accurately both the position and the velocity of a particle at the same instant.” This might require a short explanation for the physicist illiterates. Position is the identification of the relative location, in other words: where you are. Velocity is the speed and direction, in other words: where you are going. There is a short ‘true story’ that illustrates this. It is rumored that Heisenberg went for a drive one day and got stopped by a traffic cop for speeding. The cop asked, “Do you know how fast you were going?” and Heisenberg replied, “No, but I know where I am.”
The same seems to apply for many organizations today. They know where they are (or have been) but often they do not know where they are going. The main reason for this is that their (big) data is at rest. It is mostly inactive data that is stored physically in any digital form, for example a database or a data warehouse. Also it is used primarily for historic reporting or analysis on mostly internal data done by the IT department. Although the quality is high (data warehouses for example are often associated with a high level of data quality and creation of the single version of the truth), the time-to-market is often low (batch-oriented overnight architectures) and the value of the data is therefore relatively low.
To be truly successful many organizations need to transform from hindsight analysts to foresight action takers. This implies that the (big) data is not in rest but in motion or even in use. It should flow in real time through the organization and change business outcomes on the fly. Not by a small team of IT experts but by everybody within the organization.
Big Data has created a paradigm shift in the way we look at decision making today. Traditionally, structured data from internal systems like ERP has been the main source for this. Now unstructured data comes from sensors in machines, planes, trains, automobiles or even your fridge. They add to the amount of data available.

But this is also the time where external data, from websites or social media, tells us much more about our own performance. Not with facts or dimensions from the IT data warehouse but with opinions from Twitter and likes on Facebook by your customers. This is the time where Facebook can predict when somebody is about to cheat or commit suicide, where Google can predict a flu outbreak, or retailers can predict that your teenage daughter is pregnant. It is about bringing outside intelligence inside your organization.
Although the name suggests otherwise, Big Data is not only about volume. Volume is about data at rest. It is about storing massive volumes of data against lower cost. However Big Data is not about where you are (position) but about where you are going (velocity) with speed as the deciding factor. Research (for example by The Economist) shows that most C-level executives are convinced that value from data can be found in real time. Supported by technology – like in-memory – business users must therefore quickly find usable insights in Big Data from inside and outside their own organization to make decisions on the fly.
To me, there is no uncertainty in that. 

This contribution by Jorgen Heizenberg 

Part of Capgemini’s TechnoVision 2014 update series. See the overview here.