Locally Invariant Explanations: Towards Stable and Unidirectional
  Explanations through Local Invariant Learning
  Locally interpretable model agnostic explanations (LIME) method is one of the
most popular methods used to explain black-box models at a per example level.
Although many variants have been proposed, few provide a simple way to produce
high fidelity explanations that are also stable and intuitive. In this work, we
provide a novel perspective by proposing a model agnostic local explanation
method inspired by the invariant risk minimization (IRM) principle --
originally proposed for (global) out-of-distribution generalization -- to
provide such high fidelity explanations that are also stable and unidirectional
across nearby examples. Our method is based on a game theoretic formulation
where we theoretically show that our approach has a strong tendency to
eliminate features where the gradient of the black-box function abruptly
changes sign in the locality of the example we want to explain, while in other
cases it is more careful and will choose a more conservative (feature)
attribution, a behavior which can be highly desirable for recourse.
Empirically, we show on tabular, image and text data that the quality of our
explanations with neighborhoods formed using random perturbations are much
better than LIME and in some cases even comparable to other methods that use
realistic neighbors sampled from the data manifold. This is desirable given
that learning a manifold to either create realistic neighbors or to project
explanations is typically expensive or may even be impossible. Moreover, our
algorithm is simple and efficient to train, and can ascertain stable input
features for local decisions of a black-box without access to side information
such as a (partial) causal graph as has been seen in some recent works.
