Accounting for Model Uncertainty in Algorithmic Discrimination
  Traditional approaches to ensure group fairness in algorithmic decision
making aim to equalize ``total'' error rates for different subgroups in the
population. In contrast, we argue that the fairness approaches should instead
focus only on equalizing errors arising due to model uncertainty (a.k.a
epistemic uncertainty), caused due to lack of knowledge about the best model or
due to lack of data. In other words, our proposal calls for ignoring the errors
that occur due to uncertainty inherent in the data, i.e., aleatoric
uncertainty. We draw a connection between predictive multiplicity and model
uncertainty and argue that the techniques from predictive multiplicity could be
used to identify errors made due to model uncertainty. We propose scalable
convex proxies to come up with classifiers that exhibit predictive multiplicity
and empirically show that our methods are comparable in performance and up to
four orders of magnitude faster than the current state-of-the-art. We further
propose methods to achieve our goal of equalizing group error rates arising due
to model uncertainty in algorithmic decision making and demonstrate the
effectiveness of these methods using synthetic and real-world datasets.
