FACT: A Diagnostic for Group Fairness Trade-offs
  Group fairness, a class of fairness notions that measure how different groups
of individuals are treated differently according to their protected attributes,
has been shown to conflict with one another, often with a necessary cost in
loss of model's predictive performance. We propose a general diagnostic that
enables systematic characterization of these trade-offs in group fairness. We
observe that the majority of group fairness notions can be expressed via the
fairness-confusion tensor, which is the confusion matrix split according to the
protected attribute values. We frame several optimization problems that
directly optimize both accuracy and fairness objectives over the elements of
this tensor, which yield a general perspective for understanding multiple
trade-offs including group fairness incompatibilities. It also suggests an
alternate post-processing method for designing fair classifiers. On synthetic
and real datasets, we demonstrate the use cases of our diagnostic, particularly
on understanding the trade-off landscape between accuracy and fairness.
