Interpreting Black Box Models via Hypothesis Testing
  In science and medicine, model interpretations may be reported as discoveries
of natural phenomena or used to guide patient treatments. In such high-stakes
tasks, false discoveries may lead investigators astray. These applications
would therefore benefit from control over the finite-sample error rate of
interpretations. We reframe black box model interpretability as a multiple
hypothesis testing problem. The task is to discover "important" features by
testing whether the model prediction is significantly different from what would
be expected if the features were replaced with uninformative counterfactuals.
We propose two testing methods: one that provably controls the false discovery
rate but which is not yet feasible for large-scale applications, and an
approximate testing method which can be applied to real-world data sets. In
simulation, both tests have high power relative to existing interpretability
methods. When applied to state-of-the-art vision and language models, the
framework selects features that intuitively explain model predictions. The
resulting explanations have the additional advantage that they are themselves
easy to interpret.
