Counterfactual Risk Assessments, Evaluation, and Fairness
  Algorithmic risk assessments are increasingly used to help humans make
decisions in high-stakes settings, such as medicine, criminal justice and
education. In each of these cases, the purpose of the risk assessment tool is
to inform actions, such as medical treatments or release conditions, often with
the aim of reducing the likelihood of an adverse event such as hospital
readmission or recidivism. Problematically, most tools are trained and
evaluated on historical data in which the outcomes observed depend on the
historical decision-making policy. These tools thus reflect risk under the
historical policy, rather than under the different decision options that the
tool is intended to inform. Even when tools are constructed to predict risk
under a specific decision, they are often improperly evaluated as predictors of
the target outcome.
  Focusing on the evaluation task, in this paper we define counterfactual
analogues of common predictive performance and algorithmic fairness metrics
that we argue are better suited for the decision-making context. We introduce a
new method for estimating the proposed metrics using doubly robust estimation.
We provide theoretical results that show that only under strong conditions can
fairness according to the standard metric and the counterfactual metric
simultaneously hold. Consequently, fairness-promoting methods that target
parity in a standard fairness metric may --- and as we show empirically, do ---
induce greater imbalance in the counterfactual analogue. We provide empirical
comparisons on both synthetic data and a real world child welfare dataset to
demonstrate how the proposed method improves upon standard practice.
