Interpreting Multivariate Shapley Interactions in DNNs
  This paper aims to explain deep neural networks (DNNs) from the perspective
of multivariate interactions. In this paper, we define and quantify the
significance of interactions among multiple input variables of the DNN. Input
variables with strong interactions usually form a coalition and reflect
prototype features, which are memorized and used by the DNN for inference. We
define the significance of interactions based on the Shapley value, which is
designed to assign the attribution value of each input variable to the
inference. We have conducted experiments with various DNNs. Experimental
results have demonstrated the effectiveness of the proposed method.
