Maintaining fairness across distribution shift: do we have viable
  solutions for real-world applications?
  Fairness and robustness are often considered as orthogonal dimensions when
evaluating machine learning models. However, recent work has revealed
interactions between fairness and robustness, showing that fairness properties
are not necessarily maintained under distribution shift. In healthcare
settings, this can result in e.g. a model that performs fairly according to a
selected metric in "hospital A" showing unfairness when deployed in "hospital
B". While a nascent field has emerged to develop provable fair and robust
models, it typically relies on strong assumptions about the shift, limiting its
impact for real-world applications. In this work, we explore the settings in
which recently proposed mitigation strategies are applicable by referring to a
causal framing. Using examples of predictive models in dermatology and
electronic health records, we show that real-world applications are complex and
often invalidate the assumptions of such methods. Our work hence highlights
technical, practical, and engineering gaps that prevent the development of
robustly fair machine learning models for real-world applications. Finally, we
discuss potential remedies at each step of the machine learning pipeline.
