How do you ensure you can trust software?

We go about our everyday lives without fully understanding the systems that keep us safe and secure. That is because safety is vested in software. Gone are the days of knowing how your car works. Modern vehicles require around 100 million lines of software to make them work. Of course, most of the software runs the navigation and entertainment systems and heats the seats. However, some of that software is tasked with keeping us safe, such as making sure the braking and engine management systems are working.

But how can we know for sure that the software is doing its job? The short answer is, we can’t. So we have no choice but to trust the people who designed and built it.

Moving critical functionality from hardware into software is a well-trodden path as complex industries mature. Arguably, the aviation industry was the original pioneer with fly-by-wire systems as far back as the 1960s. It also drove early work in programming language design and international software quality standards.

A rocky road

Of course, the pace of change has accelerated in the last fifty years. Today’s critical software functionality that ensures your safety and security is not just in planes and cars. It is in medical devices, home automation, the electrical grid, gas meters, etc.

The transition to trusted software has not always been smooth. Consider these three examples:

• Between 2010 and 2014, over 700 UK postal service employees were prosecuted, and some went to jail for fraud because the new computer system was adding up the number incorrectly
• Toyota’s well-documented US safety issues with its car braking system in 2014 resulted in a $1.2 billion criminal penalty
• In 2018, the UK National Health Service reported that computer software kills between 100 to 900 people a year in the UK alone

The fact that software plays a crucial role in keeping us safe is not disputed. Yet cases like these give the perception that software cannot be written without bugs. It’s just too hard to make it work all the time. Consumers still get told to “turn it off and on again” to clear the problem. And for companies with software problems, this excuse can be the screen they hide behind.

But it’s a myth. A false perception that needs correcting. For many reasons, the software industry has not led the way in correcting this false perception. Now, as can be seen from the examples above, law courts around the world are challenging best practices in software production.

The way forward

It has been argued that a solution to this problem is to make all source code “open” and thus available for independent scrutiny. Consider the Heartbleed bug in the security library used for online internet transactions. Despite being used by millions of people, the bug went undetected in open software. This shows us that just because something is open, it’s not necessarily bug-free. Software quality is an orthogonal topic to software ownership, visibility, or business model.

The good news is that we know how to use engineering discipline and rigor to write correct software. Software that you can rely on to keep you secure and alive. When automotive companies say it is inevitable that autonomous cars will have many software faults, I point to air travel. Air traffic control, autopilot, automatic landing systems, etc., perform their jobs daily with minimal fuss because the software does exactly what it was designed and written to do.

The Capgemini Engineering approach

Capgemini Engineering has over 35 years of experience building software systems for demanding can’t-fail environments across industries as diverse as air traffic control, aircraft avionics, defense, railways signaling and train control, nuclear and renewable power generation, and banking and finance. The common theme across these sectors is that whatever the function, and whoever the end-user, the software must work first-time every time.

Cross-pollination of good ideas from one industry to another is an integral part of the solution, with aviation leading the way. Using the right processes and tools is equally important. It is essential to train and empower staff to understand the implications of the software they write and their responsibilities.

Into the future

As manufacturers realize the technical complexity of their products has moved from hardware to software, they will need to step up their game. But, for industries with little regulation, consumers will drive accountability through the courts.

Rather than all industries starting from ground zero, manufacturers can look to the aviation industry for guidance. No other industry has the depth and breadth to produce high-quality software. We should not be shy about our achievements. We know that applying engineering discipline to software development produces a reliable product.

The hard reality today is that software is responsible for our safety. So, we need to make sure we build the software correctly. The aerospace industry has a lot of mature processes, tools, and culture that it can share with sectors that are new to the challenge of producing safety-related software.

Capgemini Engineering is proud of its track record in taking these tools and processes from aerospace and adapting them to work in other industries.

Author: Neil White, Director, High Integrity Software, Capgemini Engineering

Neil has over 25 years of experience building software systems for environments that cannot tolerate failure. He has worked in industries as diverse as aviation, power generation, railway infrastructure, defense, and banking security.