Threading the Needle of On and Off-Manifold Value Functions for Shapley
  Explanations
  A popular explainable AI (XAI) approach to quantify feature importance of a
given model is via Shapley values. These Shapley values arose in cooperative
games, and hence a critical ingredient to compute these in an XAI context is a
so-called value function, that computes the "value" of a subset of features,
and which connects machine learning models to cooperative games. There are many
possible choices for such value functions, which broadly fall into two
categories: on-manifold and off-manifold value functions, which take an
observational and an interventional viewpoint respectively. Both these classes
however have their respective flaws, where on-manifold value functions violate
key axiomatic properties and are computationally expensive, while off-manifold
value functions pay less heed to the data manifold and evaluate the model on
regions for which it wasn't trained. Thus, there is no consensus on which class
of value functions to use. In this paper, we show that in addition to these
existing issues, both classes of value functions are prone to adversarial
manipulations on low density regions. We formalize the desiderata of value
functions that respect both the model and the data manifold in a set of axioms
and are robust to perturbation on off-manifold regions, and show that there
exists a unique value function that satisfies these axioms, which we term the
Joint Baseline value function, and the resulting Shapley value the Joint
Baseline Shapley (JBshap), and validate the effectiveness of JBshap in
experiments.
