Is Shapley Explanation for a model unique?
  Shapley value has recently become a popular way to explain the predictions of
complex and simple machine learning models. This paper is discusses the factors
that influence Shapley value. In particular, we explore the relationship
between the distribution of a feature and its Shapley value. We extend our
analysis by discussing the difference that arises in Shapley explanation for
different predicted outcomes from the same model. Our assessment is that
Shapley value for particular feature not only depends on its expected mean but
on other moments as well such as variance and there are disagreements for
baseline prediction, disagreements for signs and most important feature for
different outcomes such as probability, log odds, and binary decision generated
using same linear probability model (logit/probit). These disagreements not
only stay for local explainability but also affect the global feature
importance. We conclude that there is no unique Shapley explanation for a given
model. It varies with model outcome (Probability/Log-odds/binary decision such
as accept vs reject) and hence model application.
