What is the situation with legacy systems and practices? Can this hold back businesses when it comes to making the most of their data?
The most common practices we see that hold organizations back are when businesses have not embarked on building a modern data platform. A modern data platform must be business value driven and provide trusted data, by design, from event to effective action. It must be repeatable and extendable – and therefore scalable. This is really hard to do and some of the main challenges include dealing with multiple legacy systems covering ERP (enterprise resource planning) such as Oracle, SAP Peoplesoft, which help people manage their assets, procurement processes, projects, HR, etc. Legacy systems can also be CRM (customer relationship management) such as Siebel, Salesforce, etc. which help organizations manage and profile their customers to enable them to analyze and drive up loyalty and purchasing and retention.
These systems, which typically cost millions to implement and support for FTSE 250 businesses or public sector bodies, are essential but in many cases so complex and difficult to get data in and out to join up that it makes it very challenging for businesses to be able to denote cause and effect between an action and reaction. They usually come with their own reporting system, which comes with best practice dashboards but only show what is in the single system. Most large organizations have multiple ERP and CRM systems which have come about through merger and acquisition activity or organically.
The difficulty of joining up the data between systems and of having common definitions and meaning of data is holding back progress. The unique identifier of a customer in the CRM system rarely matches the unique identifier of the customer in the billing system or credit control system. Therefore, organizations end up being limited to asking the questions of the data that they have access to instead of being able to ask the questions that could drive meaningful change. For example, without a linked set of data, how can you find out what the net lifetime value of a customer is if you include marketing spend and cost of acquisition? Which customers are profitable and which are loss making? When do you break even?
The recent drive to Software-as-a-Service solutions has actually made this problem even harder. So-called legacy systems were typically running on servers owned by the organization and their IT teams could get access to all the data in the ERP and CRM systems and could build data warehouses to allow them to join the data and to perform analytics. The recent trend for cloud-based SaaS services such as Oracle Fusion, Salesforce, MS Dynamic ERP, and CRM means that the system is run and managed by the vendor for you but it also means you do not have access under the hood to all the data. Data needs to be extracted through APIs and can be costly and slow.
How aware are businesses of this issue? Do they accept there is a problem?
“Businesses are highly aware of how difficult it is to join up data, get common definitions, analyze cause and effect, and predict the future.
“It is not optional to manage data and GDPR regulations mean all businesses have to have good data management practices to avoid fines. The GDPR (General Data Protection Regulation) sets a maximum fine of €20 million (about £17.8 million) or 4% of annual global turnover – whichever is greater – for infringements.
“This change is here to stay, though, and the increasing drive to the cloud and the pay-as-you-go model and Opex over Capex is also here to stay. The key is what to do about it and whether the investment is put in to build the tools and organization change.”
How can businesses go about updating these? What skills and investment would they require to build platforms and processes that can help them make better use of data?
Most organizations we see are migrating from the expensive, on-premises, monolithic systems to more agile, cloud-based, Software-as-a-Service packages to manage their processes. Our most frequent conversations with customers are about how to migrate from the old to the new cloud-based technology.
Enabling organizations to make better use of data requires four things:
- Building a modern data platform that enables agile delivery and supports self-service, reporting, and analytics. When a global soft drink manufacturer wanted to increase its insights and analytics capabilities and make AI viable at scale, we helped establish a single data lake for organization-wide use. Now, data is always available to business users within a 30-minute timeframe. The solution’s capabilities are being extended to a range of business units and to external partners, such as bottlers. As a result, the client is now much better positioned to realize business value through collaboration around data.
- Adopt a data-driven culture and Mindset. Traditional methods are too slow to deliver value. Collaboration across key roles involved in delivery of data pipelines and analytics must be driven up. Streamlining the flow from requirements definition, development and tracking value
- Invest in data trust services. Data management means enabling a strong data catalogue to ensure people know what is available, embedding data quality practices into the platform, automating data lifecycle management so only the most useful data is used, having a strong reference and master data management solution to allow data to be joined between systems and data privacy and security.
- Focusing on automation. Data management teams need modern interoperable tools to acquire, organize, prepare, and analyze/visualize data. With large organizations having hundreds and sometimes thousands of frequently changing data sources, tools that can unify data using machine learning, apply fuzzy matching algorithms to identify patterns, and apply data quality rules are becoming more and more important and are maturing.
Enabling artificial intelligence and realizing that the patterns in the data are far too complex for a human to identify and to use AI to drive insights.
What can organizations do to develop consistency in their working practices while being flexible in adapting to different environments, and technological or economic changes?
“The key to the success of such an initiative is the creation of the correct operating model and organization structure to define and embed best practices into the different parts of the company.
“In 2001, the analyst firm Gartner started recommending that organizations create BICCs (BI competency centers). A BICC would coordinate the activities and resources for an organization. It has responsibility for the governance structure for BI and analytical programs, projects, practices, software, and architecture.
“In recent years, this has transformed to analytics competency centers (ACCs). ACCs follows a more strategic objective and follow the strategic objective to transform the company towards a data-driven company, build analytics expertise, formulate a data strategy, identify use cases for data mining, establish a manage a platform and drive the general adoption of analytics across the organization. The focus of an ACC is the adoption of self service and empowering the business. The combination of an ACC and DataOps provide the way to ingest, unify and analyze the data for modern organizations.”
How is this likely to shape up in the future? How important is it that companies adapt and evolve, and what are the risks for those that can’t?
Not being data driven is quite simply not an option in the digital economy. We have all seen how industries can be completely transformed by new digitally native entrants. Look at Uber, Airbnb. We have all seen how these digitally native businesses have transformed customer expectations as well. Look at how Amazon Prime allowing next day or same day delivery has made people used to that service and how we now expect the same from other legacy businesses.
“COVID-19 has accelerated the decline of the high street and the consumer shift to online. Fifty-nine percent of consumers worldwide said they had high levels of interaction with physical stores before COVID-19, but today less than a quarter (24%) see themselves in that high-interaction category.”
Managing the supply chain, driving down costs, managing human capital and assets are all critical to the survival of legacy businesses. Data must be treated as an asset and managed accordingly. Businesses that fail to meet this challenge will have a higher cost base and be less competitive than businesses that do.
For more info, reach out to the author, Giles Cuthbert, Head of Microsoft Data and Visualisation Practice, Insights and Data UK.