The future of medical care is software – part 2

This blog continues from Part 1: Software as a Medical Device is helping transform the healthcare sector into an intelligent industry. Here I discuss critical aspects of SaMD verification and validation (V&V).

In the medical world, there is a clear distinction between verification and validation. Verification is the process of ensuring that the design outputs meet the design inputs. At the same time, validation is the process of ensuring that the medical software or device meets the intended purpose.

The International Medical Device Regulators Forum (IMDRF), an industry governing body, has developed very specific regulations for the clinical validation of SaMD. Over the last few years, many of the emerging digital testing technologies have been verification activities. Below, we look first at SaMD validation and then SaMD verification.

Validation for SaMD

According to the IMDRF document IMDRF/SaMD WG/N41 FINAL: 2017, the process of performing a clinical evaluation has changed significantly for SaMD in that there should be a valid clinical association between the output of a SaMD and the targeted clinical condition. (See Figure 1.) Also, SaMD should provide expected technical and clinical data.
Source: International Medical Device Regulators Forum

A valid clinical association is an indicator of the level of clinical acceptance and how much meaning and confidence can be assigned to the clinical significance of the SaMD’s output in the intended healthcare situation and the clinical condition/physiological state.

Analytical validation measures the ability of an SaMD to accurately, reliably, and precisely generate the intended technical output from the input data.

Clinical validation measures the ability of an SaMD to yield a clinically meaningful output associated with the target use of SaMD output in the target healthcare situation or condition identified in the SaMD definition statement.

Being able to generate evidence to demonstrate the valid clinical association, analytical validation, and clinical validation of a SaMD is essential for establishing the SaMD’s value for users. The purpose of the assessment of the evidence is to select information based on its merits and limitations to demonstrate that the clinical evaluation evidence is high-quality, relevant, and supportive of the SaMD’s intended use.

Verification for SaMD

While SaMD can improve health outcomes using data, many SaMD builders face underlying fundamental testing challenges. The biggest challenge involves integrating modern product-development methodologies – designed for an ever-evolving internet-of-everything world – where patient safety is paramount.

Also, the host of new Android and iOS devices on which the medical software can work brings other challenges such as device compatibility testing, upgrades, backward compatibility testing, etc. Since many of these companies also deliver updates by rapid mass distribution over the internet, tracking lifecycle-related aspects poses a challenge to strict governing bodies such as the US Food and Drug Administration (FDA). There are many other challenges ahead for SaMD, including usability, performance, maintainability, and security.

The right approach

There has been a tremendous boost from the DevOps and Agile methodologies of testing that resolve many of these challenges. While not a new concept, DevOps principles have gained a tremendous amount of support given their consistent ability to reduce the time from development to operations. Agile development promotes collaboration among small teams to make quick decisions and ensure continuous delivery.

Quality assurance (QA) can sometimes be a bottleneck, but quality engineering introduces mobile app testing and automation earlier in the process. This has led to the rise of the software development engineer in test (SDET), who is responsible not only for writing the code but for testing and fixing the code as well. The implications for verification are clear from this. Testing has become iterative in nature, with quick turnout times for test cycles. It has also led to the rise of new methodologies such as test-driven development (TDD) and behavior-driven development (BDD).

There has been a gradual shift from performance testing to performance engineering. Performance engineering ensures the components of your network are functioning as intended. It offers testing teams more flexibility, data, and better opportunities to automate processes. The implications for verification are clear from this. Since performance becomes a critical measuring metric throughout the software development lifecycle (SDLC), the focus on meeting performance and other non-functional requirements becomes paramount and is tested at every stage of the SDLC.

When cost and market readiness are important, mobile app testing automation is imperative. Automated testing helps teams make the most of their testing resources and frees up test engineers, allowing them to focus on tests that require manual attention and human interaction. Many of the apps are also based on the microservices-based architecture, and application programming interface (API) testing has also become significant.

Also important is the increased adoption of open-source tools. Open-source testing tools are extremely versatile. They cover testing for web apps and all mobile app types: native, web, and hybrid. Additionally, most open-source platforms offer code libraries for any programming language. Many are customizable and adaptable to changes in the technological landscape. The critical thing to note here is that the governing agencies require companies to perform software intended use validation (SIUV) and software of unknown provenance (SOUP) validation as necessary. While commercial off-the-shelf tools might have many of these test reports generated, the onus of validating them lies squarely with the medical device companies that adopt these open-source tools.

While many of these tools and trends are being included in the verification of the SaMD ecosystem, many of the challenges are being mitigated in the fast-paced world of SaMD. The techniques mentioned above help in the V&V of SaMD and ensure patient safety and regulatory compliance.

RELATED ARTICLE:

Next Generation healthcare – Software as a medical device (SAMD) in the intelligent industry – Part 1 of series

Vivek Jaykrishnan

Vice President – Head of Industrialization Assets & IP