Does enforcing fairness mitigate biases caused by subpopulation shift?
  Many instances of algorithmic bias are caused by subpopulation shifts. For
example, ML models often perform worse on demographic groups that are
underrepresented in the training data. In this paper, we study whether
enforcing algorithmic fairness during training improves the performance of the
trained model in the \emph{target domain}. On one hand, we conceive scenarios
in which enforcing fairness does not improve performance in the target domain.
In fact, it may even harm performance. On the other hand, we derive necessary
and sufficient conditions under which enforcing algorithmic fairness leads to
the Bayes model in the target domain. We also illustrate the practical
implications of our theoretical results in simulations and on real data.
