To continue from my previous blog Next-generation healthcare – Software as a Medical Device (SaMD) in the Intelligent Industry, I will discuss the critical aspects of verification and validation of SaMD here.
In the medical world, there is a clear distinction between verification and validation. Verification is the process of ensuring that the design outputs meet the design inputs and validation is the process of ensuring that the medical software or device meets the intended purpose.
The governing body International Medical Device Regulators Forum (IMDRF) has come up with very specific regulations for clinical validation of SaMD. Many of the emerging technologies over the last few years for digital testing fall under the purview of verification activities. In the sections that follow, we will look at SaMD validation and verification.
Validation for SaMD
As per the IMDRF document (IMDRF/SaMD WG/N41 FINAL: 2017), the process of performing a clinical evaluation has significantly changed for SaMD, there should be a valid clinical association between the output of a SaMD and the targeted clinical condition. Also, SaMD should provide expected technical and clinical data.
A valid clinical association is an indicator of the level of clinical acceptance and how much meaning and confidence can be assigned to the clinical significance of the SaMD’s output in the intended healthcare situation and the clinical condition/physiological state.
Analytical validation measures the ability of a SaMD to accurately, reliably, and precisely generate the intended technical output from the input data.
Clinical validation measures the ability of a SaMD to yield a clinically meaningful output associated with the target use of SaMD output in the target health care situation or condition identified in the SaMD definition statement.
Being able to generate evidence to demonstrate the valid clinical association, analytical validation, and clinical validation of a SaMD is essential for establishing the SaMD’s value for users. The purpose of the assessment of the evidence is to select information based on its merits and limitations to demonstrate that the clinical evaluation evidence is high-quality, relevant, and supportive of the SaMD’s intended use.
Verification for SaMD
While SaMD is able to improve health outcomes using data, there are some underlying fundamental testing challenges faced by many SaMD builders. The biggest challenge involves integrating modern product development methodology — designed for an ever-evolving Internet of Everything world — with patient safety being paramount.
Also, with a whole new host of Android and iOS devices on which the medical software can work, it brings various other challenges such as device compatibility testing, upgrades, and backward compatibility testing, etc. Since many of these companies also deliver updates by mass and rapid distribution over the internet, tracking lifecycle-related aspects poses a challenge to a strict governing body such as the FDA. Apart from these, there are many other aspects, including usability, performance, maintainability, and security that come into the picture for SaMD.
The right approach
To resolve many of these challenges, there has been a tremendous boost to the DevOps and the agile methodology methods of testing. While not a new concept, DevOps principles have gained a tremendous amount of support with their consistent ability to reduce the amount of time from development to operations. Agile development promotes collaboration among various small teams to make quick and continuous delivery.
As QA can sometimes be a bottleneck, quality engineering allows the introduction of mobile app testing and automation earlier in the process. This has led to the rise of the Software Development Engineer in Test (SDET) role. The implications for verification are clear from this. Testing has come iteratively in nature, with quick turnout times for test cycles. It has also led to the rise of new methodologies such as Test-Driven Development (TDD) and Behavior-Driven Development (BDD).
There has been a gradual shift from performance testing to performance engineering. Performance engineering ensures the components of your network are functioning as intended. It offers testing teams more flexibility, data, and better opportunities to automate processes. The implications for verification are clear from this. Since performance becomes a key measuring metric right through the SDLC, the focus on meeting performance and other non-functional requirements becomes paramount and is tested at every stage of the SDLC.
When cost and market readiness are important, mobile app testing automation is imperative. Automated testing helps teams make the most of their testing resources and frees up test engineers, allowing them to focus on tests that require manual attention and human interaction. Many of the apps are also based on the microservices-based architecture and the Application Programming Interface (API) testing has also become significant.
There is also increased adoption of open-source tools. Open-source testing tools are extremely versatile. They cover testing for web apps and all mobile app types: native, web, and hybrid. Additionally, most open-source platforms offer code libraries for any programming language. They are also often customizable and adaptable to changes within the technological landscape. The important thing to note here is that the governing agencies require that companies perform Software Intended Use Validation (SIUV) and Software Of Unknown Provenance (SOUP) validation as necessary. While commercial off-the-shelf tools might have many of these test reports generated, the onus on validating these squarely lies with the medical device companies adopting these open source tools.
With many of these latest trends being adopted into the verification of the SaMD ecosystem, many of the above-mentioned challenges are mitigated in the fast-paced world of SaMD. The techniques mentioned above help in the verification and validation of SaMD and ensure patient safety and regulatory compliance.