EDF Energy and Capgemini: Independent Testing on a Crucial Smart Meter Program

The requirement: IT testing on a multi-partner program

EDF Energy’s smart meter rollout is a large, complex program, and one with a high public profile. It is a part of a government initiative that will see smart meters deployed to the entire UK population by 2020, enabling customers to take control of their energy use and costs.

To support its rollout program, which will affect over five million customers, EDF Energy needed to build a range of IT systems: for example, to manage the meters, carry out billing, and interact with the rest of the industry.

The company chose a multi-partner approach for the IT development and implementation work, engaging a best-in-class supplier for each aspect. Capgemini was appointed to run integration testing. The brief was to make sure that the outputs produced by the other partners worked properly, both individually and together. The overall aim was a smooth end-to-end customer journey, from making the appointment to have the meter fitted through to installation and beyond.

Extract from EDF Energy’s original Request for Proposal

Key principles:

• Do it once, do it right: avoiding unnecessary cost and waste and with a positive customer experience.
• Health and Safety: looking after the welfare of staff and customers.
• Transparency: sharing information, risks and issues so that we are all clear about matters that affect the operational and financial aspects of the rollout.
• Predictability and integration: delivering on time, especially where success is dependent upon other partners or EDF Energy activity.
• Resolving issues: at the point of identification, thus allowing the rollout to proceed.
• Change management: supporting the evolution of the EDF Energy smart meter rollout and the management of uncertainty that may come from the wider GB smart and governmental energy agenda.
• Shared incentives: creating financial incentives that encourage the desired behaviours and reward joint working.
• One team mentality and approach: thinking and operating as one team to deliver the EDF Energy smart metering rollout.
• Using our best collective talent on the right activities: employing the best possible staff to deliver the rollout.

We believe that success of the deployment is encapsulated in the following statement: “Doing the basics brilliantly”.

Approach to testing

Risk-based testing. To achieve the optimum balance of costeffectiveness and thorough test coverage, Capgemini has been using a risk-based approach, in accordance with the Test Management Approach (TMap^®), Capgemini Group’s business-driven methodology for structured software testing. This methodology is designed to address the key issues of quality, time and cost across the whole development lifecycle of solution delivery.

Risk-based testing involves collaborating with the business to identify the scenarios that should be tested according to their likelihood of occurrence and the probable impact should they occur. This approach ensures that testing effort is allocated in line with the needs of the business.

Addressing testing and quality from project initiation onwards. A key principle for the project was establishing test criteria and identifying tests to be performed as early as possible, in line with Capgemini’s PointZERO® approach. This avoids spending effort on rework, and helps to create business solutions that are fit for purpose.

Innovative commercial model. Capgemini has put 15% of its fee at risk by making it dependent on a series of key performance indicators (KPIs). The most important of these are:

1. Successfully uncovering problems that the development parties subsequently accept as genuine problems. If more than one in 10 defects found by Capgemini are rejected, a portion of the fee is forfeited, but results have consistently been well below the 10% threshold (at 7% for one release and 4% for another).
2. Minimising the number of defects found after the system has gone live. If more than 10% of these are categorized as being of a type that should have been found in testing, a portion of the fee would be lost. Once again, this has never happened.
3. How well the Capgemini team performs in terms of collaboration and adding value. This criterion has also been met throughout the program.

Rightshore® staffing. Initially all work was done onshore, but gradually the balance has shifted to a mixture of onshore and offshore working – an optimal arrangement from the point of view of cost and efficiency. Many of the Indian staff involved have spent a period working in the UK, which helps to ensure everyone works as one team wherever they are located.

Benefits of collaboration with Capgemini

Capgemini has been working on the program since 2013. It has contributed to three completed releases so far. EDF Energy has already rolled out meters to selected customers, ensuring that the meters, and all the associated technology including IT systems, work as they should. This means that when mass deployment starts there should be few surprises for customers.

As independent test partner, Capgemini has been able to help EDF Energy orchestrate a complex ecosystem and implement an intricate operational model for IT. Independence from design and build positions Capgemini to give EDF Energy an objective view of overall progress on the project.

Capgemini’s work is acknowledged to have been extremely successful, with thorough testing ensuring that defects get fixed well before they can affect staff or customers, which makes for smooth and uneventful releases.

“Our smart meter program has been built around a number of small releases. Following each release, we have what’s called a ‘storm period’, where our partners give us extra support to solve any issues rapidly. Our first three storms have been very light, thanks to our systems integrators, and importantly to our integration test partner, Capgemini.”
Julie Meanwell – Transformation Director, EDF Energy