Balancing Competing Objectives with Noisy Data: Score-Based Classifiers
  for Welfare-Aware Machine Learning
  While real-world decisions involve many competing objectives, algorithmic
decisions are often evaluated with a single objective function. In this paper,
we study algorithmic policies which explicitly trade off between a private
objective (such as profit) and a public objective (such as social welfare). We
analyze a natural class of policies which trace an empirical Pareto frontier
based on learned scores, and focus on how such decisions can be made in noisy
or data-limited regimes. Our theoretical results characterize the optimal
strategies in this class, bound the Pareto errors due to inaccuracies in the
scores, and show an equivalence between optimal strategies and a rich class of
fairness-constrained profit-maximizing policies. We then present empirical
results in two different contexts -- online content recommendation and
sustainable abalone fisheries -- to underscore the applicability of our
approach to a wide range of practical decisions. Taken together, these results
shed light on inherent trade-offs in using machine learning for decisions that
impact social welfare.
