Utilities don’t have a big data problem—they have a big problem with data

The traditional utility business model of generating and distributing energy was relatively simple: utility companies would only collect very elementary data relating to the levels of consumption on customer premises. Customer engagement only happened through the one-way communication of energy bills. With the digitization of traditional grids, featuring advanced metering infrastructure (AMI) systems and intelligent supervisory control and data acquisition (SCADA) systems coupled with the additional data generated by the deployment of “smart” technologies like smart home, smart city, and others came exponential increases in the data volumes. The addition of such smart infrastructure has led to the progression from a traditional grid with unidirectional data flow to an informed grid with enormous real-time data. For example, five million smart meters sending updates every 15 minutes result in nearly 16 terabytes of information per year—a three-thousand-fold increase in the amount of data generated. This mass of information has the potential to offer customer insights and business intelligence, but it needs to be unlocked so utilities can get additional insights.

Most utilities are still using traditional business intelligence (BI) reporting solutions such as data warehousing, dashboards, reports, and descriptive analytics, which are implemented to meet regulatory requirements, ensure accurate billing, and improve operational efficiency. However, the digitization of the grid, or utility IoT, requires an evolution in data management maturity to support the new data being collected and allow utilities to deal with clusters of information that could have an impact on the business—resulting in a hunger for additional analytics capabilities.

These new capabilities require an analytical engine that can manage this highly distributed data and provide results that can be optimized to solve business problems. Using big data technology, dissecting and analyzing data collected from the grid and from customers can result in new insights for utilities leading to operational efficiencies, new products, new customer programs, and fresh revenue streams. The data is a valuable resource that needs to be mined.

Developing a coherent data management strategy should include data governance, data quality, centralized funding for analytics, and a master data management approach. The plan may be hindered by the limitations on ROI models, regulatory requirements, scarce skill sets, siloed structures, technological complexity, and data acquisition issues. However, leveraging analytics for competitive advantage and generating additional revenue can be achieved by understanding the big data journey.

Most utilities are aware of the benefits they can gain from analyzing data from their smart grids and its customers but are unsure where to begin. The task may seem overwhelming. It is a challenge to translate large volumes of new data into true, actionable intelligence and to leverage the information gained from analysis to make business decisions. But the value is there for those willing to start.

Utilities need collaboration between the business and IT stakeholders to create a comprehensive roadmap focused on analytics with clearly defined objectives and achievable milestones. The plan should include the following steps:

• Identify and prioritize business initiatives
• Define analytics roadmap
• Define enterprise information model
• As-is study and gap analysis
• Data analytics solution design.

Utilities may have different catalysts to embark on a big data journey, but developing a roadmap is a key activity to sharpen the vision and inspire an organization to adopt a new analytics vision. The insights are only part of the journey and there will be fine-tuning required. Regulatory requirements, technology, weather, demographics, and other influences will also impact the project but the analytics journey will evolve along the path. It is a journey worth taking.

Learn more by reading: Transformation of intelligent utilities with analytics, big data, and internet of things.

Ajay Verma

Expert in Smart Energy and Smart Meter

I help customers use smart metering to reduce operating costs and implement the energy efficiency programs such as demand response.