Risk of Training Diagnostic Algorithms on Data with Demographic Bias
  One of the critical challenges in machine learning applications is to have
fair predictions. There are numerous recent examples in various domains that
convincingly show that algorithms trained with biased datasets can easily lead
to erroneous or discriminatory conclusions. This is even more crucial in
clinical applications where the predictive algorithms are designed mainly based
on a limited or given set of medical images and demographic variables such as
age, sex and race are not taken into account. In this work, we conduct a survey
of the MICCAI 2018 proceedings to investigate the common practice in medical
image analysis applications. Surprisingly, we found that papers focusing on
diagnosis rarely describe the demographics of the datasets used, and the
diagnosis is purely based on images. In order to highlight the importance of
considering the demographics in diagnosis tasks, we used a publicly available
dataset of skin lesions. We then demonstrate that a classifier with an overall
area under the curve (AUC) of 0.83 has variable performance between 0.76 and
0.91 on subgroups based on age and sex, even though the training set was
relatively balanced. Moreover, we show that it is possible to learn unbiased
features by explicitly using demographic variables in an adversarial training
setup, which leads to balanced scores per subgroups. Finally, we discuss the
implications of these results and provide recommendations for further research.
