Are skin lesion analyzers ready for prime time?

While the FDA is proposing to reclassify two Class III computer-assisted melanoma detection devices as Class II devices, most members of its General and Plastic Surgery Devices Panel suggested such a move would be premature during the Committee meeting. Medical Device Advisor on Friday.

According to the FDA, reclassification of the MelaFind and Nevisense devices should be considered because the health risks associated with these skin lesion analyzers (SLAs) can be mitigated through a series of “special controls” that “provide reasonable assurance of their safety and efficacy.”

However, the general consensus among most panel members is that there is currently not enough information available to establish those special controls.

“My concern is that these devices should show an improvement over what a dermatologist can do,” said Karla Ballman, PhD, of Weill Cornell Medicine in New York City. “Right now, I don’t think we feel comfortable stating what the sensitivity should be and what the specificity should be. And even more so, what the relationship between the two should be. You need to have agreed performance standards to be able to be a class II , and we don’t have that right now. So I don’t think we’re ready to define what the special controls should be.”

While these SLAs represent an area of ​​great need, said Maral Skelsey, MD, of the Washington Dermatologic Surgery Center in Chevy Chase, Maryland, “we’re just not ready for prime time, even with these special controls.”

Class III is the strictest regulatory category for devices, and according to the FDA, Class III devices are typically high-risk devices for which there is insufficient information to provide reasonable assurance of safety and efficacy through testing alone. general or special. In the case of a class II device, general controls would not be sufficient to ensure safety and effectiveness. However, there is enough information with these devices to establish special controls, such as performance standards or special labeling requirements, that would provide this guarantee.

Therefore, for these computer-assisted melanoma detection devices, mitigation measures can be taken to control associated risks, which could include false-negative or false-positive results, user error or misuse of the device, or failure or malfunction of the device. said Scott Kominsky, PhD, of the FDA’s Office of Surgical Devices and Infection Control. “That is, when the device is used correctly, the likely health benefits outweigh the likely risks, there is no unreasonable risk of illness or injury, and the device will provide clinically meaningful results in a significant portion of the target population.” .”

Murad Alam, MD, of Northwestern University in Chicago, noted that given the limited information available with only the two devices that have been FDA-approved, “I’m struggling with the urgency of doing this. It seems a little premature to me given the lack of clarity in this space.

According to Neil Ogden, BME, chief of the FDA’s General Surgery Devices Branch, device reclassification is not unusual. “We did this with hip and knee implants. We graded them from III to II,” he explained. “When we have enough information and we think we can regulate these devices in a lower class, we move to do it.”

That makes it easier for the FDA to obtain “high-quality devices that are safe and effective for the US population in the least expensive way,” he added.

However, in addressing the issue of risks associated with SLAs, the panel highlighted the issue of false negatives and positives as a major area of ​​concern.

For example, Skelsey noted that due to numerous false positives, “my real-world experience is that I abandoned a device because it didn’t work in a dermatology office.”

“There needs to be more testing, especially around the issue of false negatives and false positives,” said Neil Farber, MD, of the University of California, San Diego. “It’s such a big risk and such a complex issue because you have a [artificial intelligence] device on the one hand and an interaction with dermatologists on the other. I think we need some kind of real-world, real-use studies. And those would have to be prospective.”

While most panel members indicated they were not in favor of reclassifying the devices, others disagreed.

Katalin Roth, MD, JD, of George Washington University in Washington, DC, said she believed downgrading devices would be appropriate as long as special controls were clear, devices were used only by dermatologists with training and informed consent . of patients

SLA in development

While Friday’s meeting focused on the two devices that have already received FDA approval, the panel on Thursday discussed how the FDA should assess the benefits and risks of computer-assisted SLAs that are currently in development.

Much of the discussion focused on performance thresholds for devices intended for adjunctive use by dermatologists and non-dermatologists (i.e., primary care physicians), as well as performance thresholds for stand-alone devices (meaning that the result would be relied upon verbatim as a guide). management). Such a device could be used by non-professionals as well as by health professionals.

In general, the panel resisted the idea of ​​using preset thresholds to assess the performance of these devices. “No one seems comfortable with providing a general predetermined sensitivity/specificity threshold,” said panel chair Hobart W. Harris, MD, MPH, of the University of California, San Francisco, in summarizing the panel’s opinion.

“I don’t think absolute measures of sensitivity and specificity … are enough for us to consider these devices,” said Veronica Rotemberg, MD, PhD, of Memorial Sloan Kettering Cancer Center in New York City. “The best kind of test for these devices will be a prospective study in the intended use setting rather than retrospective threshold data, and in that intended use setting, improvement defined by a value judgment about the standard of care in that scene.”

“I agree that in the add-on setting, you want to see if you’re increasing accuracy, no matter how you define it, over where you currently are,” Alam said. “I’m not that concerned about FDA regulation of devices that are for adjunctive use; the FDA seems to handle that pretty well. The potential issues are these devices that are going to be in the general public and are going to be marketed very heavily.” aggressive”.

Once these devices are FDA-approved, “nobody will really think about the nuances of how they work or what they’re intended for, they’ll just say they’re FDA-approved,” he added. “So it’s a high standard for us that what comes out is reasonably good. I’d like it to be a sensitivity of at least what a dermatologist would do, and ideally a little more.”