The idea of using smartphones to detect potential skin cancers has been around for more than a decade, but until now, the technology hasn’t been advanced enough to challenge the accuracy of a trained dermatologist. Recently, however, researchers at MIT and Harvard developed deep learning algorithms that can detect skin cancer with stunning accuracy.

Thus far, algorithms built to automatically detect skin cancers so far have been trained to analyze individual skin lesions for odd features that could be indicative of melanoma, but what the researchers from MIT and Harvard have developed is a bit different. Instead, the researchers used the “ugly duckling” criteria, which is based on the concept that most moles on an individual will look similar and those that don’t, the so-called ugly ducklings, are recognized as a warning sign of melanoma.

The researchers say their system is the first of its kind to replicate this process, and they started by building a database of more than 33,000 wide-field images containing not just a patient’s skin, but other objects and backgrounds. The team brought the ugly duckling method into the mix by building 3D maps of all the lesions in a given image – spread across a patient’s back, for example – and ran calculations on how odd the features were on each lesion. By comparing how unusual some of these features were compared to those on other lesions in the image, the system is able to assign values and determine which ones were dangerous.

The technology was put to the test recently to identify suspicious lesions in 68 different patients using 135 wide-field photos. Individual lesions were given an oddness score based on how concerning their features were, with the assessments compared to those made by three trained dermatologists. The algorithm agreed with the consensus of the dermatologists 88 percent of the time, and the individual dermatologists 86 percent of the time.

“This high level of consensus between artificial intelligence and human clinicians is an important advance in this field, because dermatologists’ agreement with each other is typically very high, around 90 percent,” says co-author Jim Collins. “Essentially, we’ve been able to achieve dermatologist-level accuracy in diagnosing potential skin cancer lesions from images that can be taken by anybody with a smartphone, which opens up huge potential for finding and treating melanoma earlier.”