Artificial intelligence (AI) can accurately predict the race of patients from medical images alone, according to a study by the Massachusetts Institute of Technology (MIT).
Using imaging data of chest X-rays, limb X-rays, chest CT scans, and mammograms, a team of MIT scientists trained a deep-learning model to identify race as White, Black or Asian — even though the images themselves contained no explicit mention of the patient’s race.
To help understand how the model was able to do this, researchers investigated possible mechanisms of race detection, looking at variables such as differences in anatomy, bone density, resolution of images— and many more, yet the models demonstrated a high ability to detect race from chest X-rays.
“These results were initially confusing, because the members of our research team could not come anywhere close to identifying a good proxy for this task,” said Marzyeh Ghassemi, assistant professor in the MIT Department of Electrical Engineering and Computer Science and the Institute for Medical Engineering and Science (IMES), an affiliate of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and of the MIT Jameel Clinic.
“Even when you filter medical images past where the images are recognisable as medical images at all, deep models maintain a very high performance.”
This could be because, in addition to algorithms looking at vital signs or laboratory tests, they’re also looking at race, ethnicity or sex – even if that information is hidden, according to Leo Anthony Celi, principal research scientist in IMES at MIT and associate professor of medicine at Harvard Medical School.
However, the fact that algorithms ‘see’ race could be dangerous as such “superhuman capacities are generally much more difficult to control, regulate, and prevent from harming people”, said Ghassemi. Thus, Celi believes the discovery “should make us pause and truly reconsider whether we are ready to bring AI to the bedside”.
“Just because you have representation of different groups in your algorithms, that doesn’t guarantee it won’t perpetuate or magnify existing disparities and inequities. Feeding the algorithms with more data with representation is not a panacea,” said Celi
The study, entitled AI recognition of patient race in medical imaging: a modeling study, was published in Lancet Digital Health on 11 May 2022. Celi and Ghassemi wrote the paper alongside 20 other authors in four countries. The work was supported, in part, by the National Institutes of Health.
