Skin cancer detection apps on smart phones do not not provide adequate protection to the public experts have warned after a review of the evidence found poor and variable performance.

The review in the BMJ set out to examine the validity and findings of studies looking at the accuracy of algorithm based artificially intelligent (AI) smartphone apps that offer the potential for earlier detection and treatment of suspicious moles. 

The World Health Organization estimates between 2 and 3 million non-melanoma skin cancers and 132,000 melanoma skin cancers occur globally each year, but survival is high if melanoma is spotted early, which makes prompt detection and treatment crucial.

In Europe, two apps (SkinVision and SkinScan) are currently available and are regulated as class 1 medical devices (deemed to have a low to moderate risk to the user). No apps currently have US Food and Drug Administration (FDA) approval.

False reassurance leads to delays in people seeking medical advice.

There are fears apps could be harmful, particularly if false reassurance leads to delays in people seeking medical advice. A previous expert review of such apps suggested there is a high chance of skin cancers being missed. 

Nine relevant studies that evaluated six different apps were identified. Studies were small and overall of poor quality. Problems in the studies included the suspicious moles were chosen by clinicians not the app users, the photographs were taken by trained researchers on study phones, not by users on their own phones, and photographs that could not be evaluated by the apps were excluded. Also study participants were not followed up to identify cancers which were missed by the apps.

SkinScan was evaluated in a single study of 15 moles with five melanomas. The app did not identify any of the melanomas. SkinVision was evaluated in two studies. One study of 108 moles (35 cancerous or precancerous moles) achieved a sensitivity of 88% and a specificity of 79%. This means that 12% of patients with cancerous or precancerous moles would be missed, while 21% of those non-problematic moles would be wrongly identified as potentially being cancerous.

The authors explained that this meant in a population of 1000 users in which 3% have a melanoma, SkinVision could still miss four of 30 melanomas and 200 people would wrongly be told their mole was of high concern. But they point out that the limitations of the studies suggest that more errors are likely to be made.

Professor Jon Deeks at the University of Birmingham and Professor Hywel Williams at the University of Nottingham, said: “Our review found poor and variable performance of algorithm based smartphone apps, which indicates that these apps have not yet shown sufficient promise to recommend their use."

They warn that the current regulatory processes “are inadequate for protecting the public against the risks created by using smartphone diagnostic or risk stratification apps.” And they say healthcare professionals “need to be aware of the limitations of algorithm based apps to reliably identify melanomas, and should inform potential smartphone app users about these limitations.”