Fair Performance Metric Elicitation
  What is a fair performance metric? We consider the choice of fairness metrics
through the lens of metric elicitation -- a principled framework for selecting
performance metrics that best reflect implicit preferences. The use of metric
elicitation enables a practitioner to tune the performance and fairness metrics
to the task, context, and population at hand. Specifically, we propose a novel
strategy to elicit group-fair performance metrics for multiclass classification
problems with multiple sensitive groups that also includes selecting the
trade-off between predictive performance and fairness violation. The proposed
elicitation strategy requires only relative preference feedback and is robust
to both finite sample and feedback noise.
