Some years, decades ago, we didn’t have big problems with visualizing our data. The sources were well known and the amount of data to be reported fitted in a few pages. Also the requirements of visualization were easy. Tables were fine, maybe some simple charts here and there.

But in times of Big Data, visualization of this data can become an issue.

In most scenarios the journey starts with some Data Exploration where you want to get a rough understanding about the data, the content, the structure. Here you need simple and self explaining visualizations and easy tables for getting:.

  • distinct values
  • distributions
  • rough data volumes
  • and other such information

After that you can dig deeper:

The conventional route is to create reports, mostly in PDF or Excel, where you have to answer the questions about the business performance in the last X period. This is the “classical” BI usage in former times. There you have some charts at the beginning of your report to summarize the content, followed by lists with granular details in the next ten pages or so. To generate such reports it often takes time, because of the calculations in the background, the amount of data or because there are reports inside the report.
The reports are very useful, because you need them for compliance, finances, audits, stakeholders etc.

But increasingly you need the data and answers right now. The questions are much simpler, e.g. “what where the top 5 products sold in the last 2 weeks?” or “how much did we sell each month in the last 6 months?” or something like that. For answering such questions, you need other tools, like SAP Lumira, Tableau or QlikSense. Products belonging to this category are very good for initial data exploration and ad hoc analysis. Often users ask for “self service BI tools”, which is also an attribute of these tools.

For more complex questions like “How many products of category ABC will we sell next in the 2 weeks?” you need either a crystal ball – or the reliable option, Predictive Analytics. Here you can use a lot of statistical models and algorithms to classify your customers and predict product sales, demand planning and dynamic pricing.
You should always give the “customers of Big Data” a tool they can answer questions with. And the more data we get, the more power our servers have and the better our computing models get, the more important it is to get insights from the data.

With data increasing in volume, variety and velocity, Business decision makers require real-time, actionable insights. The art or science of Data Analysis is usually left to the data scientists and PhD statisticians. For democratic access to data, and for insights from that data to be understood by all, visualization becomes significant. It is therefore important not just to look at the backend of Big Data. This calls for a paradigm shift in BI front-ends. But more on that in my next blog…