Deep learning models of artificial intelligence are now able to identify someone’s race just from their X-rays.
An international team of health researchers from the US, Canada, and Taiwan came up with this application of technology.
“We aimed to conduct a comprehensive evaluation of the ability of AI to recognize a patient’s racial identity from medical images,” write the researchers in their published paper.
“We show that standard AI deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities, which was sustained under external validation conditions.”
This paper builds on the previous finding which stated that artificial intelligence scans of X-ray images were more likely to miss signs of illness in Black people.
It’s possible that the system is finding signs of melanin, the pigment that gives skin its color, that is as yet unknown to science.
“Our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging,” write the researchers.
The research demonstrates how the AI systems can reflect the biases and prejudices of human beings, whether that’s racism, sexism, or something else. Skewed training data can lead to skewed results, making them less useful.
It is an important limitation to consider as AI is the future.
“We need to take a pause,” research scientist and physician Leo Anthony Celi from the Massachusetts Institute of Technology told the Boston Globe.
“We cannot rush bringing the algorithms to hospitals and clinics until we’re sure they’re not making racist decisions or sexist decisions.”
The research has been published in The Lancet Digital Health.