FORML: Learning to Reweight Data for Fairness
  Deployed machine learning models are evaluated by multiple metrics beyond
accuracy, such as fairness and robustness. However, such models are typically
trained to minimize the average loss for a single metric, which is typically a
proxy for accuracy. Training to optimize a single metric leaves these models
prone to fairness violations, especially when the population of sub-groups in
the training data are imbalanced. This work addresses the challenge of jointly
optimizing fairness and predictive performance in the multi-class
classification setting by introducing Fairness Optimized Reweighting via
Meta-Learning (FORML), a training algorithm that balances fairness constraints
and accuracy by jointly optimizing training sample weights and a neural
network's parameters. The approach increases fairness by learning to weight
each training datum's contribution to the loss according to its impact on
reducing fairness violations, balancing the contributions from both over- and
under-represented sub-groups. We empirically validate FORML on a range of
benchmark and real-world classification datasets and show that our approach
improves equality of opportunity fairness criteria over existing
state-of-the-art reweighting methods by approximately 1% on image
classification tasks and by approximately 5% on a face attribute prediction
task. This improvement is achieved without pre-processing data or
post-processing model outputs, without learning an additional weighting
function, and while maintaining accuracy on the original predictive metric.
