Tilted Empirical Risk Minimization
  Empirical risk minimization (ERM) is typically designed to perform well on
the average loss, which can result in estimators that are sensitive to
outliers, generalize poorly, or treat subgroups unfairly. While many methods
aim to address these problems individually, in this work, we explore them
through a unified framework -- tilted empirical risk minimization (TERM). In
particular, we show that it is possible to flexibly tune the impact of
individual losses through a straightforward extension to ERM using a
hyperparameter called the tilt. We provide several interpretations of the
resulting framework: We show that TERM can increase or decrease the influence
of outliers, respectively, to enable fairness or robustness; has
variance-reduction properties that can benefit generalization; and can be
viewed as a smooth approximation to a superquantile method. We develop batch
and stochastic first-order optimization methods for solving TERM, and show that
the problem can be efficiently solved relative to common alternatives. Finally,
we demonstrate that TERM can be used for a multitude of applications, such as
enforcing fairness between subgroups, mitigating the effect of outliers, and
handling class imbalance. TERM is not only competitive with existing solutions
tailored to these individual problems, but can also enable entirely new
applications, such as simultaneously addressing outliers and promoting
fairness.
