Business users are being driven to look beyond a unified corporate view by the need for agility and quick access to insights. Enterprise Data Warehouse (EDW) environments have failed to catch up. But, IT departments are generally reluctant to adopt initiatives that threaten to disrupt status-quo.
In theory, EDWs are structured to produce a unified corporate view. A typical EDW comprises multiple layers of information aimed at transforming data into a format that is suitable for reporting. To optimally leverage the EDW environment, business users and IT are expected to focus on data accuracy and structure. As a result EDWs can never come up to speed with diversity of data. This obsession with structured, known, internal data is not just limiting in scope. It’s also time-consuming. The net result is a sub-optimal, canonical view of business information, out-of-sync with the needs of a modern, federated enterprise that seeks access to real-time insights.
Data delayed is opportunity denied, and this is a situation business users don’t like to be in. So, the ability to enforce a unified corporate view is greatly reduced when business users resort to localized, legacy alternatives.
Business Data Lake is a new approach to providing data to all constituents of the enterprise, consolidating existing data marts to satisfy reporting and information management requirements. It uses a distributed file system to replace the source loading layer along with a combination of technologies to recreate and simplify other layers. This results in a greatly improved version of business information, while extending the features of the existing EDW.
Should you worry about Data Lake rendering existing EDWs redundant?No. On the contrary, the data lake significantly reduces costs and adds years to the life of EDWs. In fact, as an information management platform, the Business Data Lake is more universal and extends the functionalities of the EDW environment. What it renders obsolete is the IT-driven culture of unduly long lead times for business insights, which is a big plus for the business users.
When scaled-up efficiently, traditional EDWs are capable of handling the large volumes of data. This approach, however, significantly increases the total costs of storage. For your Big Data needs, a Business Data Lake can take on Petabytes of data and analyze it for outcomes at lightning speeds. It can incorporate new Big Data sources, including unstructured data, and deliver it to several users across the enterprise – all this, at reduced total costs.
So what’s unsettling IT?
Governance, ROI, and performance are generally cited as risks with the Big Data Lake approach. But you’ll see in my subsequent posts that these are largely unfounded and that Business Data Lakes can flawlessly co-exist with EDWs.
Share with us how business users in your organizations are driving platform and governance discussions on data management.