In my previous article Master Data is the “Noun” in Big Data “Sentences” – Part 1 we discussed the challenges of adding business context to the Big Data world or the Digital Universe and thereby losing out on potential big opportunities. In this part we will discuss potential solutions to the problem.
So the problem is: How do you add context to the Digital Universe? My idea is very simple — if you want to make Big Data Useful to your organization then you need to link it to Master Data.
So how do you translate my Idea into practice? Big Data has variety, volume and velocity, Master Data is structured and complex. There are two possible situations that arises from trying to associate Master Data and Big Data.
- You have linked Big Data Identifiers (BDIs) with your Master Data and thus you can use your BDIs as the linkage between Master Data and Big Data.
- You have no common linkage between Master Data and Big Data.
The first case is the easier of the two to handle. During the Data Ingestion process i.e. loading of data from your Big Data sources into the Big Data repository we need to identify Big Data Identifiers. Big Data Identifiers are basically your social network user names or identifiers which are available as part of the feeds from various social networks, machines ids or mac ids from various sensors and machine logs. These BDIs could then be co-related back to the master data. Using this relationship master data context could be added to your analysis. For example if you are able to identify the product that is generating maximum noise in the market you should then be able to filter the noise to understand positive or negative responses. These sentiments could then be used to create an action plan that may trigger news items, campaigns, product improvements, recall, etc. If you are able to add the customer dimension to that same analysis you may be further able to filter the noise to get better and deeper perspective of what your ‘Real’ customers think about your product instead of generalizing the response or sentiment. This is only possible by linking Master Data to the Big Data
The second case is the more common of the two and the more complex to address. In this case we lack direct co-relation between the master data and the various data feeds because of lack of known BDIs. This is where my solution will be highly applicable. You need to think of Big Data as “Sentences” and then identify the “Nouns” in these sentences. Once you have been able to identify the “nouns” in these “sentences” we should then be able to establish the relationship between the two, the “Verb”, which is nothing but action. The “Verb” can then lead to meaningful analysis or action.
On the Master Data side we can identify “nouns” through various associations, relationships, purchases, interactions, campaigns etc. We can then use these nouns as common interlinks to connect master data to the Big Data. These “nouns” can serve as the dimension for our analysis. Such kind of analysis has been conducted in the past too – like grouping and segmentation.
However in the Digital Universe this require a new set of perspectives and learning algorithms to identify and classify these “nouns”. It is important to build the right taxonomy and the hierarchy. This is where the MDM Solution can play a critical role. Master Data Management solution within your organization can serve as the starting point to build and identify the “Nouns” or terms for your taxonomy. These terms can then be used to tag big data feeds.
“Nouns” identified by the learning algorithms can also be tagged back to the Master Data as appropriate. Master Data along with the taxonomy can then serve as one of the dimensions of analysis and thereby produce more meaningful and actionable analysis.
As our data changes into information and needs to be transformed into knowledge – our ways and approach towards creating, organizing, managing and analysis information needs to change. This is one of the many possible approaches. Information Management needs to evolve and fast to meet the needs of the new age Digital Organizations.