Thriving On Data #1 – My Data Is Bigger Than Yours

The rise in volume (amount of data), velocity (speed of data) and variety (range of data, both structured and unstructured) gives way to new information architectures and tools that no longer only collect and store but actually use data to create business value and provide usable insights. Start to imagine what is possible when you are able to handle much more data than ever before, from sources inside and outside the organization that were previously unthinkable, integrated and available in near real-time.

Often, the Big Data phenomenon is first seen as a constraint, an issue, a new problem to solve. But Big Data is there; your business and of course your IT are already impacted. It is a fact. It is time to make the most out of it, using Big Data to improve customer experiences, to increase market share, to reduce time to market, to propose new services, to develop new business models.

How? Well, under this new Big Data, there is a secret sauce. This new and massive data is not coming alone; it is coming with some new technologies, more particularly:

– Hadoop: This open source and free software allows you to store and analyze very high volumes of data at very limited costs. So limited that, in many cases, we should consider that Hadoop stores data for free (compared with other storage pricing, it is a negligible cost).

– In-memory: Working in-memory (so without hard disk, except for reliability) means an incredible performance improvement, often between 100 and 1,000 times faster than standard applications. Data discovery and analytics are becoming so fast that we can say Business Intelligence is now ready for immediate, agile use more than ever.

– Event Processing: This concept is about usable insights: every time something happens, you are able to react to create value (e.g., pushing the right product in order to increase the shopping basket value) or to increase user experience (predictive maintenance, fast reaction on social networks after a customer complaint). So your operations can be guided, automated and optimized in real-time.

– Cloud: All these new Big Data technologies are available from the Cloud. Among other things, this means that procurement cycles shorten dramatically, hence resulting in a very short time to market for solutions. But the other interesting aspect is that Hadoop, in-memory and event processing solutions have been designed from the start to work from the Cloud and are ready for dynamic scale in both processing and storage: you pay only what you really need.

When joining these 4 innovations, it appears we are able to thrive on data without limits: as much data as needed, as fast as needed, on time and at optimized cost. The real border that now needs to be crossed is rethinking your business, taking into account IT is no longer a limiting factor but instead makes the unthinkable possible.

Now that’s what you call ‘Big.’

This contribution by Manuel Sevilla 

Part of Capgemini’s TechnoVision 2014 update series. See the overview here.