FairLens: Auditing Black-box Clinical Decision Support Systems
  The pervasive application of algorithmic decision-making is raising concerns
on the risk of unintended bias in AI systems deployed in critical settings such
as healthcare. The detection and mitigation of biased models is a very delicate
task which should be tackled with care and involving domain experts in the
loop. In this paper we introduce FairLens, a methodology for discovering and
explaining biases. We show how our tool can be used to audit a fictional
commercial black-box model acting as a clinical decision support system. In
this scenario, the healthcare facility experts can use FairLens on their own
historical data to discover the model's biases before incorporating it into the
clinical decision flow. FairLens first stratifies the available patient data
according to attributes such as age, ethnicity, gender and insurance; it then
assesses the model performance on such subgroups of patients identifying those
in need of expert evaluation. Finally, building on recent state-of-the-art XAI
(eXplainable Artificial Intelligence) techniques, FairLens explains which
elements in patients' clinical history drive the model error in the selected
subgroup. Therefore, FairLens allows experts to investigate whether to trust
the model and to spotlight group-specific biases that might constitute
potential fairness issues.
