Write It Like You See It: Detectable Differences in Clinical Notes by Race Lead to Differential Model Recommendations

Hammaad Adam, Ming Ying Yang, Kenrick Cato, Ioana Baldini, Charles Senteio, Leo Anthony Celi, Jiaming Zeng, Moninder Singh, Marzyeh Ghassemi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

Clinical notes are becoming an increasingly important data source for machine learning (ML) applications in healthcare. Prior research has shown that deploying ML models can perpetuate existing biases against racial minorities, as bias can be implicitly embedded in data. In this study, we investigate the level of implicit race information available to ML models and human experts and the implications of model-detectable differences in clinical notes. Our work makes three key contributions. First, we find that models can identify patient self-reported race from clinical notes even when the notes are stripped of explicit indicators of race. Second, we determine that human experts are not able to accurately predict patient race from the same redacted clinical notes. Finally, we demonstrate the potential harm of this implicit information in a simulation study, and show that models trained on these race-redacted clinical notes can still perpetuate existing biases in clinical treatment decisions.

Original languageEnglish (US)
Title of host publicationAIES 2022 - Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
PublisherAssociation for Computing Machinery, Inc
Pages7-21
Number of pages15
ISBN (Electronic)9781450392471
DOIs
StatePublished - Jul 26 2022
Event5th AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, AIES 2022 - Oxford, United Kingdom
Duration: Aug 1 2022Aug 3 2022

Publication series

NameAIES 2022 - Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

Conference

Conference5th AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, AIES 2022
Country/TerritoryUnited Kingdom
CityOxford
Period8/1/228/3/22

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Social Sciences (miscellaneous)

Keywords

  • clinical notes
  • health equity
  • natural language processing

Fingerprint

Dive into the research topics of 'Write It Like You See It: Detectable Differences in Clinical Notes by Race Lead to Differential Model Recommendations'. Together they form a unique fingerprint.

Cite this