Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods
  For AI systems to garner widespread public acceptance, we must develop
methods capable of explaining the decisions of black-box models such as neural
networks. In this work, we identify two issues of current explanatory methods.
First, we show that two prevalent perspectives on explanations ---
feature-additivity and feature-selection --- lead to fundamentally different
instance-wise explanations. In the literature, explainers from different
perspectives are currently being directly compared, despite their distinct
explanation goals. The second issue is that current post-hoc explainers are
either validated under simplistic scenarios (on simple models such as linear
regression, or on models trained on syntactic datasets), or, when applied to
real-world neural networks, explainers are commonly validated under the
assumption that the learned models behave reasonably. However, neural networks
often rely on unreasonable correlations, even when producing correct decisions.
We introduce a verification framework for explanatory methods under the
feature-selection perspective. Our framework is based on a non-trivial neural
network architecture trained on a real-world task, and for which we are able to
provide guarantees on its inner workings. We validate the efficacy of our
evaluation by showing the failure modes of current explainers. We aim for this
framework to provide a publicly available, off-the-shelf evaluation when the
feature-selection perspective on explanations is needed.
