On the Choice of Fairness: Finding Representative Fairness Metrics for a
  Given Context
  It is of critical importance to be aware of the historical discrimination
embedded in the data and to consider a fairness measure to reduce bias
throughout the predictive modeling pipeline. Various notions of fairness have
been defined, though choosing an appropriate metric is cumbersome. Trade-offs
and impossibility theorems make such selection even more complicated and
controversial. In practice, users (perhaps regular data scientists) should
understand each of the measures and (if possible) manually explore the
combinatorial space of different measures before they can decide which
combination is preferred based on the context, the use case, and regulations.
  To alleviate the burden of selecting fairness notions for consideration, we
propose a framework that automatically discovers the correlations and
trade-offs between different pairs of measures for a given context. Our
framework dramatically reduces the exploration space by finding a small subset
of measures that represent others and highlighting the trade-offs between them.
This allows users to view unfairness from various perspectives that might
otherwise be ignored due to the sheer size of the exploration space. We
showcase the validity of the proposal using comprehensive experiments on
real-world benchmark data sets.
