Image Courtesy: Getty Images
Chinese researchers have tested a new model of healthcare in which artificial intelligence is built into nearly every stage of an eye clinic, from pre-consultation and diagnosis to follow-up care. Human doctors remained involved, but the system was designed around AI from the beginning rather than adding AI tools to an existing workflow.
The clinic, known as the AI-Agent Augmented Tsinghua Eye Clinic, or AI-TEC, was developed by researchers associated with Tsinghua University and the Beijing Visual Science and Translational Eye Research Institute.
One of the clearest lessons involved the quality of the data used to train the system. The AI initially struggled to identify conditions including glaucoma and age-related macular degeneration from eye scans. Its performance improved substantially after researchers added 1,426 high-quality images that had been carefully reviewed and labeled by ophthalmologists.
Those images proved more useful than a much larger collection of nearly 27,000 lower-quality images with less comprehensive labeling. Using the improved training data, the system achieved an AUROC score above 0.93, putting its diagnostic performance roughly in line with other advanced retinal imaging systems.
The experiment also revealed a less obvious problem: even a capable AI system is unlikely to be useful if doctors find it cumbersome to operate. Five months into the trial, AI-TEC was used in only 41 of 1,113 examinations, or 3.8 percent.
After researchers simplified the system, reducing the number of clicks and the amount of manual input required, usage rose to 259 of 1,126 examinations, or 23 percent, the following month.
The researchers concluded that deploying AI in healthcare is not simply an exercise in building a more accurate algorithm. Data quality, workflow design, clinician involvement, governance and rapid feedback all play important roles.
There is also a fundamental difference between how AI and doctors approach patients. AI may focus on whether a scan indicates a particular disease, while physicians begin with the patient’s symptoms and broader clinical situation. A highly accurate scan-reading system therefore does not automatically translate into better patient care.
The study suggests that future evaluations of medical AI should look beyond benchmark scores and ask whether the technology actually improves clinical workflows and patient outcomes.
