Interpreting and Understanding Graph Convolutional Neural Network using
  Gradient-based Attribution Method
  To solve the problem that convolutional neural networks (CNNs) are difficult
to process non-grid type relational data like graphs, Kipf et al. proposed a
graph convolutional neural network (GCN). The core idea of the GCN is to
perform two-fold informational fusion for each node in a given graph during
each iteration: the fusion of graph structure information and the fusion of
node feature dimensions. Because of the characteristic of the combinatorial
generalizations, GCN has been widely used in the fields of scene semantic
relationship analysis, natural language processing and few-shot learning etc.
However, due to its two-fold informational fusion involves mathematical
irreversible calculations, it is hard to explain the decision reason for the
prediction of the each node classification. Unfortunately, most of the existing
attribution analysis methods concentrate on the models like CNNs, which are
utilized to process grid-like data. It is difficult to apply those analysis
methods to the GCN directly. It is because compared with the independence among
CNNs input data, there is correlation between the GCN input data. This
resulting in the existing attribution analysis methods can only obtain the
partial model contribution from the central node features to the final decision
of the GCN, but ignores the other model contribution from central node features
and its neighbor nodes features to that decision. To this end, we propose a
gradient attribution analysis method for the GCN called Node Attribution Method
(NAM), which can get the model contribution from not only the central node but
also its neighbor nodes to the GCN output. We also propose the Node Importance
Visualization (NIV) method to visualize the central node and its neighbor nodes
based on the value of the contribution...
