Group-based Fair Learning Leads to Counter-intuitive Predictions
  A number of machine learning (ML) methods have been proposed recently to
maximize model predictive accuracy while enforcing notions of group parity or
fairness across sub-populations. We propose a desirable property for these
procedures, slack-consistency: For any individual, the predictions of the model
should be monotonic with respect to allowed slack (i.e., maximum allowed
group-parity violation). Such monotonicity can be useful for individuals to
understand the impact of enforcing fairness on their predictions. Surprisingly,
we find that standard ML methods for enforcing fairness violate this basic
property. Moreover, this undesirable behavior arises in situations agnostic to
the complexity of the underlying model or approximate optimizations, suggesting
that the simple act of incorporating a constraint can lead to drastically
unintended behavior in ML. We present a simple theoretical method for enforcing
slack-consistency, while encouraging further discussions on the unintended
behaviors potentially induced when enforcing group-based parity.
