From Human Explanation to Model Interpretability: A Framework Based on
  Weight of Evidence
  We take inspiration from the study of human explanation to inform the design
and evaluation of interpretability methods in machine learning. First, we
survey the literature on human explanation in philosophy, cognitive science,
and the social sciences, and propose a list of design principles for
machine-generated explanations that are meaningful to humans. Using the concept
of weight of evidence from information theory, we develop a method for
generating explanations that adhere to these principles. We show that this
method can be adapted to handle high-dimensional, multi-class settings,
yielding a flexible framework for generating explanations. We demonstrate that
these explanations can be estimated accurately from finite samples and are
robust to small perturbations of the inputs. We also evaluate our method
through a qualitative user study with machine learning practitioners, where we
observe that the resulting explanations are usable despite some participants
struggling with background concepts like prior class probabilities. Finally, we
conclude by surfacing~design~implications for interpretability tools in
general.
