A Bandit-Based Algorithm for Fairness-Aware Hyperparameter Optimization
  Considerable research effort has been guided towards algorithmic fairness but
there is still no major breakthrough. In practice, an exhaustive search over
all possible techniques and hyperparameters is needed to find optimal
fairness-accuracy trade-offs. Hence, coupled with the lack of tools for ML
practitioners, real-world adoption of bias reduction methods is still scarce.
To address this, we present Fairband, a bandit-based fairness-aware
hyperparameter optimization (HO) algorithm. Fairband is conceptually simple,
resource-efficient, easy to implement, and agnostic to both the objective
metrics, model types and the hyperparameter space being explored. Moreover, by
introducing fairness notions into HO, we enable seamless and efficient
integration of fairness objectives into real-world ML pipelines. We compare
Fairband with popular HO methods on four real-world decision-making datasets.
We show that Fairband can efficiently navigate the fairness-accuracy trade-off
through hyperparameter optimization. Furthermore, without extra training cost,
it consistently finds configurations attaining substantially improved fairness
at a comparatively small decrease in predictive accuracy.
