Empirical observation of negligible fairness-accuracy trade-offs in
  machine learning for public policy
  Growing use of machine learning in policy and social impact settings have
raised concerns for fairness implications, especially for racial minorities.
These concerns have generated considerable interest among machine learning and
artificial intelligence researchers, who have developed new methods and
established theoretical bounds for improving fairness, focusing on the source
data, regularization and model training, or post-hoc adjustments to model
scores. However, little work has studied the practical trade-offs between
fairness and accuracy in real-world settings to understand how these bounds and
methods translate into policy choices and impact on society. Our empirical
study fills this gap by investigating the impact of mitigating disparities on
accuracy, focusing on the common context of using machine learning to inform
benefit allocation in resource-constrained programs across education, mental
health, criminal justice, and housing safety. Here we describe applied work in
which we find fairness-accuracy trade-offs to be negligible in practice. In
each setting studied, explicitly focusing on achieving equity and using our
proposed post-hoc disparity mitigation methods, fairness was substantially
improved without sacrificing accuracy. This observation was robust across
policy contexts studied, scale of resources available for intervention, time,
and relative size of the protected groups. These empirical results challenge a
commonly held assumption that reducing disparities either requires accepting an
appreciable drop in accuracy or the development of novel, complex methods,
making reducing disparities in these applications more practical.
