The Food and Drug Administration is seeking feedback on how to regulate medical devices that incorporate generative artificial intelligence. The agency published a discussion paper this week as it considers the “unique risks” and potential promise of the technology for patient care.
The FDA’s Center for Devices and Radiological Health has authorized more than 1,000 devices that use AI, but most do not incorporate generative AI. This technology can mimic patterns to create text, images, or audio. While the agency said the technology holds promise for care, it may introduce different challenges compared to traditional software.
The CDRH published the discussion paper on Tuesday outlining a potential framework. The document does not represent new policy or guidance, but it gives the device center the opportunity to receive feedback. Stakeholders can submit comments through October 19.
Related: Siemens, Philips win ARPA-H grants for stroke robots
The device center has sought stakeholder feedback in recent years on generative AI as the agency determines its regulatory approach. The Digital Health Advisory Committee met for the first time in 2024 to discuss total product lifecycle considerations for these devices. The committee met again last year to talk about digital mental health devices that use the technology. So far, the center has not proposed any new policies specific to generative AI.
Defining Risk and Testing Standards
One challenge is that generative AI-enabled devices have unique characteristics that make them more difficult to regulate than other types of software or AI. They may accept open-ended inputs, perform multiple tasks, and produce variable outputs. Because of this, the devices may also carry the risk of “hallucinations” that appear authentic.
In a discussion paper, the CDRH outlines three main concepts: a risk framework, a competency-based approach to premarket evaluation, and ideas for postmarket monitoring. The risk framework considers the type of activity a device performs. For example, software that provides a risk score predicting a future cardiovascular event would fall under the lowest risk category of informational, non-directive action.
Related: FDA Sets Guidelines for Wearable Technology
AI features that direct patients or clinicians to take a specific action increase in risk, with fully autonomous devices being the highest risk category. The paper acknowledged that the risk level may also depend on the circumstances. A device that autonomously prescribes antibiotics for a strep throat infection may carry a different risk of harm than a device that starts thrombolytic therapy as part of a stroke workflow.
As the CDRH thinks about how to review generative AI devices before they go to market, the device center outlined a competency-based approach. Devices would be benchmarked and then tested in a clinical setting to evaluate how the final, user-facing version performs in terms of safety and effectiveness.
Manufacturers would be responsible for postmarket monitoring, although clinicians, healthcare institutions, payers, and other authorities may also have a role to play. The device center said it is considering whether it would be appropriate to accept “greater premarket uncertainty” about the benefits and risks of a generative AI-enabled device through more reliance on postmarket monitoring.
