Feature Removal Is a Unifying Principle for Model Explanation Methods
  Researchers have proposed a wide variety of model explanation approaches, but
it remains unclear how most methods are related or when one method is
preferable to another. We examine the literature and find that many methods are
based on a shared principle of explaining by removing - essentially, measuring
the impact of removing sets of features from a model. These methods vary in
several respects, so we develop a framework for removal-based explanations that
characterizes each method along three dimensions: 1) how the method removes
features, 2) what model behavior the method explains, and 3) how the method
summarizes each feature's influence. Our framework unifies 25 existing methods,
including several of the most widely used approaches (SHAP, LIME, Meaningful
Perturbations, permutation tests). Exposing the fundamental similarities
between these methods empowers users to reason about which tools to use and
suggests promising directions for ongoing research in model explainability.
