Google AI better than human doctors at diagnosing rashes from pictures

Chatbot can analyse health-care imagery, such as PDFs of test results, to accurately diagnose a range of medical conditions

Google’s medical chatbot can use patient information and photos to pinpoint the cause of a rash. Credit: Adapted from Getty

An upgraded version of Google’s medical chatbot can use smartphone photos to diagnose rashes and can evaluate a host of other types of medical imagery — improving the bot’s ability to pinpoint the cause of ailments.

An earlier version of the artificial-intelligence (AI) system outperformed physicians in diagnostic accuracy and bedside manner. The upgrade also did better than human doctors at interpreting images such as electrocardiograms and PDFs of laboratory results.

The latest version of the system, which is called the Articulate Medical Intelligence Explorer (AMIE) and is still purely experimental, was described in a paper¹ posted to the arXiv preprint server 6 May. It has not yet been peer reviewed. The development of systems that integrate images and clinical information “brings us closer to an AI assistant that mirrors how a clinician actually thinks”, says Eleni Linos, director of the Stanford University Center for Digital Health in California, who was not involved in the research.

Health-care simulation

To test AMIE’s new capabilities, 25 actors playing patients had virtual consultations with the chatbot and a human primary-care physician. The actors simulated 105 medical scenarios involving a range of symptoms and medical histories. They also presented relevant medical images.

After each consultation, both AMIE and the human doctor suggested a diagnosis and a treatment plan. A team of 18 specialists in dermatology, cardiology and internal medicine evaluated the performances of the chatbot and the human doctors by analysing transcripts of the visits and the post-visit reports.

They found that, in general, AMIE achieved greater accuracy in its diagnoses than did the human doctors. Furthermore, its accuracy was less affected by problems such as low-quality images.

Medical school for an AI doctor

The updated version of the chatbot is based on Google’s large language model (LLM) Gemini 2.0 Flash, which can process images. The researchers adapted it for medical purposes, by adding an algorithm to improve the LLM’s ability to conduct diagnostic conversations and clinical reasoning.

To confirm that the resulting system was indeed superior to the unaltered version of Gemini 2.0, researchers commanded the model to simulate patient–physician dialogues, in which the model plays the parts of a patient, a physician and a third party that evaluates conversations between the other two. “This way, you can sort of imbue it with the right, desirable behaviours when conducting a diagnostic conversation,” says Ryutaro Tanno, a scientist at Google DeepMind in London and a co-author of the study.