Debiasing classifiers: is reality at variance with expectation?
  We present an empirical study of debiasing methods for classifiers, showing
that debiasers often fail in practice to generalize out-of-sample, and can in
fact make fairness worse rather than better. A rigorous evaluation of the
debiasing treatment effect requires extensive cross-validation beyond what is
usually done. We demonstrate that this phenomenon can be explained as a
consequence of bias-variance trade-off, with an increase in variance
necessitated by imposing a fairness constraint. Follow-up experiments validate
the theoretical prediction that the estimation variance depends strongly on the
base rates of the protected class. Considering fairness--performance trade-offs
justifies the counterintuitive notion that partial debiasing can actually yield
better results in practice on out-of-sample data.
