Meaningfully Debugging Model Mistakes using Conceptual Counterfactual
  Explanations
  Understanding and explaining the mistakes made by trained models is critical
to many machine learning objectives, such as improving robustness, addressing
concept drift, and mitigating biases. However, this is often an ad hoc process
that involves manually looking at the model's mistakes on many test samples and
guessing at the underlying reasons for those incorrect predictions. In this
paper, we propose a systematic approach, conceptual counterfactual explanations
(CCE), that explains why a classifier makes a mistake on a particular test
sample(s) in terms of human-understandable concepts (e.g. this zebra is
misclassified as a dog because of faint stripes). We base CCE on two prior
ideas: counterfactual explanations and concept activation vectors, and validate
our approach on well-known pretrained models, showing that it explains the
models' mistakes meaningfully. In addition, for new models trained on data with
spurious correlations, CCE accurately identifies the spurious correlation as
the cause of model mistakes from a single misclassified test sample. On two
challenging medical applications, CCE generated useful insights, confirmed by
clinicians, into biases and mistakes the model makes in real-world settings.
