Algorithmic encoding of protected characteristics in image-based models
  for disease detection
  It has been rightfully emphasized that the use of AI for clinical decision
making could amplify health disparities. A machine learning model may pick up
undesirable correlations, for example, between a patient's racial identity and
clinical outcome. Such correlations are often present in (historical) data used
for model development. There has been an increase in studies reporting biases
in image-based disease detection models. Besides the scarcity of data from
underserved populations, very little is known about how these biases are
encoded and how one may reduce or even remove disparate performance. There are
concerns that an algorithm may recognize patient characteristics such as
biological sex or racial identity, and then directly or indirectly use this
information when making predictions. But it remains unclear how we can
establish whether such information is actually used. This article aims to shed
some light on these issues by exploring different methodology for assessing the
inner working of disease detection models. We explore multitask learning and
model inspection to assess the relationship between protected characteristics
and prediction of disease. We believe our analysis framework could provide
valuable insights in future studies in medical imaging AI. Our findings also
call for further research to better understand the underlying causes of
performance disparities.
