Monitor data proactively to increase model resilience

Publish date:

A new approach can keep your predictive models a step ahead – even during a crisis such as a pandemic.

During the pandemic, some companies have questioned the value of their predictive models: The predictions can seem to bear no relation to reality. The companies are also not sure how to use the models during the recovery period, because it seems that data collected during the crisis is too disordered to generate accurate predictions.

Look at the data, not the model

The term “model drift” is often used to describe this problem, but I prefer “concept drift” or, especially, “data drift”, because the issue arises from data rather than from the model itself. The disordered data collected during a crisis bears no relation to the data with which the model has been trained.

To ensure that models can continue to deliver value during a crisis, Capgemini has developed an approach that separates the disordered data into two sets. One set can be analyzed to understand what has been going on during the crisis. The other set can be used by the model to predict what will happen during the recovery period.

Example: Predicting consumer demand during and after a crisis

A retailer selling healthcare products had built a model to forecast demand. During the pandemic, demand for certain products such as hand sanitizer and painkillers suddenly soared, while demand for travel-related products such as insect repellent and sun lotion plummeted. The model’s earlier predictions had little validity and nor could the retailer see how to predict demand in the recovery period. With our approach, the retailer was able to separate out the COVID-related “noise” in the data from the data that could be used to predict the future. This meant the model could continue to provide useful predictions both during and after the crisis.

As well as ensuring the model keeps working during a crisis, this approach has another major benefit: It can be used proactively to make the model resilient. Monitoring data continuously using automated processes gives better results than the usual approach of checking retrospectively for divergence between predictions and actual outcomes. It’s also more cost-effective than regular tuning regardless of need.

To find out how your model can always deliver value, even in a crisis, contact the authors:
Padmashree Shagrithaya, VP & Head – Analytics, AI and Visualization
Chandrasekhar Balasubramanyam, VP – Insights and Data

Related Posts

AI & Analytics

Generative language models and the future of AI

Rajeswaran Viswanathan
Date icon September 15, 2021

Very large language models like GPT-3 have changed the game for solving Natural Language...

Insights and Data

What can AI strategists learn from Software Development history?

Padmashree Shagrithaya
Date icon September 15, 2021

Implement MLOps to bring all the pieces together and reap larger organization-wide value from...

Insights and Data

The superpower of autonomous AI systems

Monish Suri
Date icon September 2, 2021

Statistics dressed up as AI may provide valuable insights, perception, and correlations, but...