Is Big Data about data or processes?

Publish date:

I recently read an article by my colleague Steve Jones about collaboration. Data collaboration means that “disparate data sets are going to need to collaborate to produce a new outcome”. But from my background in Enterprise Content Management (ECM), collaboration means people working on the same set of data (or documents) to achieve a common […]

Process and data modelingI recently read an article by my colleague Steve Jones about collaboration. Data collaboration means that “disparate data sets are going to need to collaborate to produce a new outcome”. But from my background in Enterprise Content Management (ECM), collaboration means people working on the same set of data (or documents) to achieve a common goal. Like in a project.

The fact that data can collaborate, just like people, surprised me. This triggered me to think about how we should approach Big Data? Is it all about data or users?

To start off, I firmly believe that organizations can get more business benefits from Big Data by combining it with business processes or workflows and users with the right expertise.

Some twenty years ago I worked on a project for a car lease firm. The goal was to create a workflow system to support the customer related processes. I remember how we went into a meeting room where the manager vividly explained on the whiteboard how these processes worked. My colleagues went out and implemented the processes as presented. But to no avail, the system was not accepted.

The company discovered that the workforce didn’t do what the management prescribed in their processes. As a manner of speaking, each customer could be treated differently. We set out to design another system that catered for this way of working where customers came first. Offering the right data at the right moment, so the customer representative could process the case as he or she found fit.

When you don’t know how exactly data is being processed, what’s being done with the data and how data is transformed to create added value, data related projects will miss their objectives. Some say that Big Data is all about data and not about processes. That might be true, but without knowing how and when data is being used, how can you meet the real needs of the consumers of your data?

In the old IT related design methodologies, like Yourdon Systems Method (YSM) by Ed Yourdon, we ride a dual carriage way. One for data modeling, and one for process modeling. These two roads should come together in one design. In Big Data projects, we might emphasize the data part more, but can we skip the process part at all? I don’t think so.

Design Thinking

Nowadays, Design Thinking is being promoted as the way to deliver data projects. Technology companies, like IBM, Intuit and GE, focus heavily on Design Thinking as the approach for modern design in IT.

Tim Brown, IDEO’s President and CEO, described that practice best stating, “Design thinking is a human-centered approach to innovation that draws from the designer’s toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.”

What are the requirements for business success and how does data help to enable that success? Most Big Data projects are simply about interpreting and presenting data to users. However, the ability to do that effectively is something that many projects struggle to achieve.

This blog post is not to elaborate on Design Thinking. So let’s focus on the first step: Empathize. Tim Brown: “Empathy is at the heart of design. Without the understanding of what others see, feel, and experience, design is a pointless task.” To learn about the audience for whom you are designing, by observation and interview. Who is my user? Who within the org would use the product? What matters to this person?

Getting this question right will affect how you conduct the project and how you design the system. With their storylines in mind, we can design a system and present the data and analytics tuned to the tasks in the processes they need to perform.

In ECM, one of the biggest challenges is not designing the processes or workflows, but to determine what information a user needs to perform a certain task. Only when this is done right, the system will truly help the user to work more effectively. Knowing the way people work is absolutely essential.

On the other hand, some data analytics products can offer you any insight it finds. Like IBM Watson Analytics, just correlating every data element to each other. And lets the users drill down until the user has found what he or she is (not) looking for. But still, we’ve to think about the contents of datasets we want to analyze. Showing analytics, however true and beautiful, the particular user cannot relate to, is just a waste. One size really doesn’t fit all users.

Big Data is about users

Let’s take a look at a successful project: IBM Watson for Oncology. This Watson based system takes all documents on certain types of cancer and the related patient records to help in diagnostics and to propose treatments for real life patients. Watson is capable of delving through all the data and presenting new insights that even surprised the expert oncologists.

In my opinion, this is a real success story for Big Data. Why, because IBM studies the way doctors and other medical staff diagnosed and treated their patients. The system has been developed in close cooperation with these experts and users of the system.

Google is using our Big Data to give you the advertisements you might like. Amazon wants to propose products shoppers desire. IBM tries to give doctors the most effective treatment plans. All at the right moment. These Big Data initiatives are designed with specific users in mind: an advertiser, a shopper, a doctor. With their needs, wishes, and drivers in the first place.

IT projects, and Big Data projects, are all about users. And the way they work and act. That’s what makes up processes. Successful Big Data deployments are about incorporating the insights from Big Data into those mundane, day-to-day processes. Processes that, however, could not have been executed without these new insights from Big Data.

Photo Public Domain: Process and data modeling via Wikimedia

Related Posts


Beyond the AIOps hype: Part 2

Sindhu Bhaskaran
Date icon August 7, 2020

In this, the second in her blog series exploring AI for IT operations (AIOps), artificial...


Interconnected World Of Big Data and Other Technologies Redefining The Future

Sumit Kumar
Date icon July 31, 2020

Data has always been an important asset in business, however the value of data was redefined...


Beyond the AIOps hype: Part 1

Sindhu Bhaskaran
Date icon March 31, 2020

AIOps (AI-enabled IT operations) offers unprecedented opportunities to get to grips with even...