Prediction Sensitivity: Continual Audit of Counterfactual Fairness in
  Deployed Classifiers
  As AI-based systems increasingly impact many areas of our lives, auditing
these systems for fairness is an increasingly high-stakes problem. Traditional
group fairness metrics can miss discrimination against individuals and are
difficult to apply after deployment. Counterfactual fairness describes an
individualized notion of fairness but is even more challenging to evaluate
after deployment. We present prediction sensitivity, an approach for continual
audit of counterfactual fairness in deployed classifiers. Prediction
sensitivity helps answer the question: would this prediction have been
different, if this individual had belonged to a different demographic group --
for every prediction made by the deployed model. Prediction sensitivity can
leverage correlations between protected status and other features and does not
require protected status information at prediction time. Our empirical results
demonstrate that prediction sensitivity is effective for detecting violations
of counterfactual fairness.
