Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
  Many existing approaches for estimating feature importance are problematic
because they ignore or hide dependencies among features. A causal graph, which
encodes the relationships among input variables, can aid in assigning feature
importance. However, current approaches that assign credit to nodes in the
causal graph fail to explain the entire graph. In light of these limitations,
we propose Shapley Flow, a novel approach to interpreting machine learning
models. It considers the entire causal graph, and assigns credit to
\textit{edges} instead of treating nodes as the fundamental unit of credit
assignment. Shapley Flow is the unique solution to a generalization of the
Shapley value axioms to directed acyclic graphs. We demonstrate the benefit of
using Shapley Flow to reason about the impact of a model's input on its output.
In addition to maintaining insights from existing approaches, Shapley Flow
extends the flat, set-based, view prevalent in game theory based explanation
methods to a deeper, \textit{graph-based}, view. This graph-based view enables
users to understand the flow of importance through a system, and reason about
potential interventions.
