Thriving On Data #1 – My Data Is Bigger Than Yours
What if there was no limit to what data could be stored, no structure to be imposed, and no filter on what could be presented and analyzed? With ‘Big Data’ solutions becoming more mature and enterprise-ready, organizations are now increasingly – and with some urgency – looking at how to actually leverage this incredible potential. Data is already so much more than ‘just’ a corporate asset: new business models are emerging and new revenue streams are created by monetizing data with business partners and customers. However, there are many opportunities to simply improve the existing data estate as well, saving costs and adding agility. Proof of concepts have been widely successful. Now the challenge is to transform all that data into insight at the point of action, delivering real business value.
The era in which technology was a limitative factor in terms of performance of data storage, access and analysis is over. Over decades we have seen business users being frustrated by sampled or limited datasets and spaghetti architecture landscapes, where data sits in silos but is replicated in each system where it is needed to answer business questions.
The wave of Big Data technologies and platforms – including Hadoop – and the analytics solutions around can now support the widest variety of workloads and finally realize the promise of store once, use many. Multiple access and analytical frameworks – increasingly powerful and simple to use – run on top of this data foundation to democratize the access to data. It utilizes SQL-based access layers (Impala, Hive, Spark SQL, Hawq, Drill, to name a few), search capabilities and vendor-specific connectors. Yes, it’s still Big Data. But it’s also Easy Access now.
Looking at combining ‘strategic workloads’ (trends or patterns detection over a long history of data) with more ‘tactical workloads’ (continuously stream data, identify in real-time specific events and react accordingly) is a key component of Big Data-style architectures. As big as these data foundational platforms may become, integrating from day one the ability to handle streaming data ‘pipelines’ from sensors, network probes, transactions and real time analytics will tremendously improve the value enterprises can create. As enterprises can now predict and prescript how their company, their marketing team, their store manager, their sales representative should react right away, they can make a compelling difference on how it is acting, where and when it matters the most.
With the agility brought by Big Data technologies, organizations need to reposition the way they manage data and projects revolving around it. New governance models emerge where agile development with integrated teams between business and IT work together.
Big Data brings a new opportunity to break away from design processes where business teams need to justify the value of adding new datasets to a BI system, before being able to manipulate the new data and evaluate its value. Too many projects and business problems are stopped or remain unanswered because of an unjustifiable or unknown value. As a result, too many business opportunities are lost.
Setting up an exploration lab (and team), leveraging a Big Data platform is key to having success within the enterprise. It will allow the team to test out new data sets, expand its reach from just its own data to the data from outside (open data, 3rd party data), and test out entirely new analytical models. This will lead enterprises to start thinking about how Bigger Data can allow them to create new services for its customers, new business models, unexpected and unexplored data-driven partnerships.
As mind-boggling as the growth opportunities brought by Big Data can be, enterprises don’t always need deep science or radical disruptive ideas to start getting in the game. It can be as simple as repositioning the problems they are already working on and understanding how Big Data can change perspectives. Why not simply start with addressing the current state of the existing BI landscape (for example through an industrialized Data Optimization approach)? Look at the problems the business is encountering and the questions that are still unresolved and see how the Big Data lens might change things. Then set up the first new governance processes and start exploring and experimenting with data.
Sooner than you think, you might be able to tell your competitors that Your Data is Bigger Than Theirs. It actually means your business is doing better. And that’s the real big thing.
Your expert: Anne-Laure Thieullent
Part of Capgemini’s TechnoVision 2015 update series. See the overview here.