We are drowning in a sea of data. Data is coming from everything and from everywhere. Whether it is transactions from an ERP system or Point of Sale data from you local supermarket. Sensor data from high voltage electricity grids or from your car. Even human interactions like facebook tweets or your holiday pictures on Flickr contibute to this. The bottom line is that we see an enormous increase in various types of data and the frequency in which we receive this flood of petabytes. We tend to name this Big Data, but we could also call it a Big Mess or even Big Brother as there are many privacy related discussions associated with this topic. At the other side we also believe that from data comes information, knowledge or even wisdom. Research from institutes like Harvard Business Review, or analysts like Gartner or Forrester all point in the same direction. Data is the new oil. It is a productivity factor for your organization, just like labor or capital. The competitive advantage for organizations is in creating insights and actions from this data (network) faster than others. We have seem to have reached a tipping point where we belief our data (explicit knowledge, the cold facts) more than our own gut feeling (implicit knowledge, our experience). We see this in every industry from intelligence based policing (remember minority report) to fact based retailing. Although there are many barriers, like ethics, financing, finding the right resources or fixing the basic, there is also plenty of innovation, inspiration and (data) integration. At the end of 2012, standing on top of the hype cycle of Big Data, we overlook the data landscape and see the following 2013 trends emerge.
1. My Data is Bigger than Yours
The rise in volume (amount of data), velocity (speed of data) and variety (range of data) gives way to new architectures that no longer only collect and store but actually use structured and unstructured data to create business value. Integrating distributed data in real time is the big challenge. Look for technology solutions like data warehouse appliances, in-memory analytics, columnar storage and smart software solutions.
2. Inflection Point for Real Time
Even though the size of data is increasing the end users are expecting faster answers from their information environment – whether it is standard reports or navigating through to source data. In-memory technology and distributed messaging systems mark the end of batch and will allow for new business usage where speed (fast data) is the first requirement. To store, process and gain insight from Big or Open Data, on-demand or real-time virtualized architectures will replace traditional data warehouses.
3. Do It Yourself
Data exploration once was the field of a limited number of expert users but it has come a long way since. Through the democratization of information, placing data in the hands of many but still as a separate process, exploration now has become a part of our daily work. With this comes the increased need to create insight on the fly by business users instead of through standard IT centered development processes making business & IT alignment an important topic. Agile techniques like SCRUM allow for a quicker go-to-market and will be the default.
4. Google fast, Apple Easy
Just like at home, business users are expecting an engine that searches all available data (structured and unstructured, internal and external) to quickly find answers and navigate through the results to find patterns and trends using advanced or even predictive analytics. The result is a ‘consumerization’ of enterprise data. The corporate data App Store is just around the corner.
5. Eye of the Beholder
With the increased supply and demand for data it is hard to see the forest for the trees. The numbers are too big for any business user to really understand. This has led to a flood of visual displays of quantitative information like infographics or geographic information systems– a completely new way to analyze and communicate your Big Data insights.
6. Supercell of Data
The amount of available data is bigger than ever which causes a need for linear horizontal scalability. Social media supplies organizations with essential information about their customers’ opinions. Combined with actual customer behavior as captured in transactional systems, a wealth of information emerges. Cloud makes this information (hardware, software, intelligence) available as-a-service via the internet on any device. Business users want to access the data anytime and anywhere. This puts increased demand on the information system architecture and information access like mobile devices and visualization.
7. ‘Analytication’of data
Crunching the numbers or competing on analytics, data is not about volume. It is about the ability to analyze and act in real time using data from sensors, transactions or interactions, both from inside as well as outside your own organization. Data can be used to solve business problems and create a competitive advantage and improve decisions in an interconnected world. The Harvard Business Review even says that: “The data scientist is the sexiest job of the 21st century.”