Fitting for the age of big data, Gary Loveman termed his Harvard Business Review article in 2003 “Diamonds in the Data Mine”. The article describes the creation of a data-driven business model based on the analysis of customer data. For Caesars Entertainment (an American gaming cooperation) it was one of their biggest success stories in history.

Today, many companies are on the search for diamonds in the data mine. Their initial goal is to find “rough diamonds” in the form of valuable data and insights. Their ultimate goal is to transform the rough diamonds into most precious “brilliants” by utilizing valuable data and insights to their full potential.

Brilliants emerge when companies are able to implement corporate actions based on data and achieve a significant performance impact. Examples of corporate actions include the optimization of processes, creation of unique customer experiences and the implementation of innovative data-driven business models, exemplified by Amazon and Google.

The image below illustrates several application fields of big data:

The two-stage process starting with the search of rough diamonds up to the creation of brilliants sets some challenges for companies. Following a study by MIT, companies which do not master the transformation to a data-driven organization have to accept a 5-6% lower profitability than their competitors. 9 out of 10 leaders and experts are sure: big data will play a central role in decision making in the next years.

It is therefore even more important for companies to understand what capabilities they have to develop and refine to excel in today’s data-driven environment.

The stony search for rough diamonds in the data mine

Those who want to search for rough diamonds successfully and efficiently have to obey three principles:

  1. Search in the right location
  2. Exactly know and protect the environment the search is conducted in
  3. Possess the right resources and capabilities to identify and find rough diamonds

These principles can be transferred to the big data context of companies. Leading companies master these principles and are one step ahead of their competitors because of the following:

They anticipating changes

Leading companies anticipate changes in the market environment and understand the behavior and demands of their customers, suppliers and competitors. Consequently they know what value they want to create out of data and possess a clear data strategy.

Based on this fundamental principle they determine which data sources to search for and for which external data sources proactive access should be secured. Specifically in the context of increasing data volume, velocity, variety and veracity it is of key importance to differentiate between relevant and irrelevant data sources.

In the next years data from social media and the Internet of Things will become far more relevant for businesses. Also, customers are increasingly aware of the value of data: according to a Gartner report more than one million people will sell their personal data in 2016.

They know about data sensitivity

Leading companies precisely know about the highly sensitive nature of data and the associated risks. They skillfully address concerns of the public regarding data privacy and are experts in handling data privacy regulations, which vary greatly among countries.

They master internal and external data ownership wars and protect sensitive data to prevent external data breaches in times of increasing and more severe cyber attacks. Especially the latter becomes more important as the average cost per capita of a sensitive data breach increased to $154.

They have mastered the shift in IT landscapes

Leading companies have mastered the shift from complex and non-transparent legacy data and IT landscapes. They possess flexible and integrated data and IT structures with the technical capabilities to analyze Big Data fast and goal-oriented. An aligned corporate data governance is as important for leading companies as a clear understanding of the skills to develop in-house and the skills to procure from the market.

In their search for rough diamonds in their data they create interdisciplinary teams consisting of data scientists, IT experts and business experts to bundle analytical, technical and business knowledge. They present big data prototypes proactively to end users and collect valuable input for further development.

Finally they are experienced in managing portfolios of Big Data projects and are able to realize synergy effects between them.

The transformation of rough diamonds to brilliants

The value of rough diamonds is limited. Their true value only emerges when they are polished and cut with a special technique which transforms them into precious brilliants.

Setting this analogy into the big data context, the best data and insights are of limited value. Tangible performance impacts will only emerge if companies embed valuable assets built from data into their processes, products, and services, and utilize insights from big data initiatives through better decision making.

In order to increase operational efficiency, enhance customer experiences and create new data-driven business models, companies have to adapt their business processes, structures and the way in decisions are made in the organization. This often implies a change in the culture of the company which promotes the joint utilization and sharing of data.

It is important to understand that this transformation does not stop at the firm’s boundary. In times when industries converge and new partnerships between tech and non-tech companies are announced, companies have to think about the bigger picture.

Over the next few years, the sharing of data between companies will go beyond the traditional data exchange along the value chain to proactively detect inventory or sales problems. For instance, in 2016, both General Electric and Rolls-Royce, created an Internet of Things platform in cooperation with major tech vendors to give their customers an integrated end-to-end view of their connected industrial assets.

The successful transformation to an experienced diamond searcher, polisher and cutter

In the area of big data, no company is born a master. Furthermore, not every company has to reach mastery. Rather, companies should develop the right degree of data capabilities depending on their competitive environment and core competencies.

There is no silver bullet for the transformation to a data-driven company. The best path depends on the predominant corporate culture, the existing organizational structures as well as the technical and organizational capabilities. Companies which already possess a company-wide innovation and information culture with integrated IT and data landscapes are better equipped to start big data initiatives across functional areas right away. Others will likely do easier in completing isolated big data initiatives in functional areas for quick value creation.

Independent of the selected transformation path the support of the top management from early on is essential. The Chief Information Officer will play a central role in those initiatives in many companies. New roles such as the Chief Data Officer and the Chief Digital Officer can also support in this endeavor.

In conclusion, companies are in the early phases of a quickly evolving age of big data. Businesses which are able to master the challenging transformation to a data-driven company can become more productive and innovative. Companies which are already well advanced in the transformation should keep up their momentum – otherwise they risk losing their competitive advantage quickly.