EMERGO SUMMARY OF KEY POINTS:

• US regulators are fashioning new guidance on software as a medical device (SaMD) based on IMDRF principles.
• The degree of clinical evaluation and evidence required of a SaMD would depend on the function it performs.
• SaMD clinical evaluation should be able to support manufacturers' claims of safety, effectiveness and performance.

The Food and Drug Administration has issued new draft guidance based on International Medical Device Regulators Forum (IMDRF) proposals for clinical evaluation of software as a medical device.

The IMDRF proposed document, issued in August 2016, establishes clinical evaluation processes for software as a medical device, or SaMD, in order to determine safety, effectiveness and performance.

Although clinical evaluations are typically required for medical devices of all types, the IMDRF proposes additional clinical evaluation recommendations for SaMD given software’s unique characteristics such as indirect contact with patients and its operating in a “complex highly connected-interactive socio-technical environment” in which product modifications occur at a much faster rate than for more conventional devices.

Categories of SaMD determined by function

First, the IMDRF and FDA lay out three main categories or types of SaMD, explaining that the level and amount of clinical evaluation required of a particular SaMD depends largely on what type of function the product is designed to perform.

The three categories, listed in ascending order of significance in terms of whether and how the SaMD output affects healthcare decisions, are:

• Informing clinical management: These products provide information on diagnosis or treatment options, or aggregate clinical data from various sources
• Driving clinical management: These products aid in treatment, prediction or diagnosis of a disease or condition
• Treating or diagnosing: These products directly treat, diagnose, screen, prevent or mitigate diseases or conditions, and their use may lead to immediate or near-term actions by healthcare providers or users

General principles for SaMD clinical evaluations

The IMDRF proposal ties clinical evaluation to a manufacturer or developer’s broader lifecycle management effort. Such evaluations should be planned prior to being conducted on a SaMD product, and incorporate appropriate risk assessment data and processes pertaining to that product.

“SaMD clinical evaluation includes the gathering and assessment of scientific validity, analytical validity and clinical (real-world, obtained from patients) performance of a SaMD,” states the guidance. “A combination of the results of these activities generates clinical evaluation evidence for a SaMD.”

How extensively a manufacturer or developer should conduct a clinical evaluation depends on several factors: the SaMD’s underlying algorithm, that algorithm’s degree of transparency, characteristics of the SaMD’s intended use target population and intended users.

Specific guidelines for SaMD clinical evaluation

The IMDRF also lists more specific recommendations for conducting SaMD clinical evaluations that can most effectively generate clinical evidence:

• Evidence goals of clinical evaluation: Evaluation should be able to demonstrate appropriate scientific validity, analytical validity and clinical performance of a SaMD.
• Required levels of clinical evaluation: Depth and breadth of a clinical evaluation should be determined based on whether clinically accepted analytical validity standards already exist, as well as clinical performance evidence.
• SaMD scientific validity evidence: The need to generate such evidence depends on whether the link between a SaMD and a clinical condition or state is already well known. Such evidence may be derived from literature reviews, manufacturer experience data and scientific validity studies.
• Analytical validity evidence: Verification and validation efforts carried out under the manufacturer or developer’s quality management system process are needed to produce this kind of evidence.
• Clinical performance evidence: Such evidence should be generated using real or simulated data sets reflecting “real patient conditions,” according to the guidance. The SaMD manufacturer or developer should identify relevant data as well as determine how much and what type of data is necessary to demonstrate appropriate clinical performance.
• Appraising clinical evaluation evidence: Evidence should allow both the SaMD manufacturer and users to determine that the product is appropriate for its intended use; that the product performs as expected according to its intended use; that the evidence sufficiently supports the product’s safety, effectiveness and performance; and that the product’s risks are acceptably balanced by its benefits.

The FDA is seeking comment from industry on the draft guidance through mid-December 2016.