Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments:early value assessment

Background Skin cancers are some of the most common types of cancer. Dermatology services receive about 1.2 million referrals a year, but only a small minority are confirmed skin cancer. Artificial intelligence may be helpful in the diagnosis of skin cancer by identifying lesions that are or are not cancerous. Objectives To investigate the clinical and cost-effectiveness of two artificial intelligence technologies: DERM (Deep Ensemble for Recognition of Malignancy, Skin Analytics) and Moleanalyzer Pro (FotoFinder), as decision aids following a primary care referral. Methods A rapid systematic review of evidence on the two technologies was conducted. A narrative synthesis was performed, with a meta-analysis of diagnostic accuracy data. Published and unpublished cost-effectiveness evidence on the named technologies, as well as other diagnostic technologies were reviewed. A conceptual model was developed that could form the basis of a full economic evaluation. Results Four studies of DERM and two of Moleanalyzer Pro were subject to full synthesis. DERM had a sensitivity of 96.1% to detect any malignant lesion (95% confidence interval 95.4 to 96.8); at a specificity of 65.4% (95% confidence interval 64.7 to 66.1). For detecting benign lesions, the sensitivity was 71.5% (95% confidence interval 70.7 to 72.3) for a specificity of 86.2% (95% confidence interval 85.4 to 87.0). Moleanalyzer Pro had lower sensitivity, but higher specificity for detecting melanoma than face-to-face dermatologists. DERM might lead to around half of all patients being discharged without assessment by a dermatologist, but a small number of malignant lesions would be missed. Patient and clinical opinions showed substantial resistance to using artificial intelligence without any assessment of lesions by a dermatologist. No published assessments of the cost-effectiveness of the technologies were identified; three assessments related to skin cancer more broadly in a National Health Service setting were identified. These studies employed similar model structures, but the mechanism by which diagnostic accuracy influenced costs and health outcomes differed. An unpublished cost–utility model was provided by Skin Analytics. Several issues with the modelling approach were identified, particularly the mechanisms by which value is driven and how diagnostic accuracy evidence was used. The conceptual model presents an alternative approach, which aligns more closely with the National Institute for Health and Care Excellence reference case and which more appropriately characterises the long-term consequences of basal cell carcinoma. Limitations The rapid review approach meant that some relevant material may have been missed, and capacity for synthesis was limited. The proposed conceptual model does not capture non-cash benefits associated with demand on dermatologist time. An assessment of the likely budget impact and resource use could not be provided. Conclusions DERM shows promising diagnostic accuracy for triage and diagnosis of suspicious cancer lesions in selected patients referred from primary care. Its impact on the diagnostic pathway and patient care is, however, uncertain. Moleanalyzer Pro shows promising accuracy for diagnosing melanoma, but its evidence base is limited. Future work While artificial intelligence has the potential to be cost-effective for the identification of benign lesions, further research addressing the limitations in the diagnostic accuracy evidence is necessary. Without comparative evidence on the diagnostic accuracy of artificial intelligence technologies, their value will remain uncertain. Study registration This study is registered as PROSPERO CRD42023475705. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Evidence Synthesis programme (NIHR award ref: NIHR136014) and is published in full in Health Technology Assessment; Vol. 30, No. 10. See the NIHR Funding and Awards website for further award information.