GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended
  Animation
  The existing graph neural networks (GNNs) based on the spectral graph
convolutional operator have been criticized for its performance degradation,
which is especially common for the models with deep architectures. In this
paper, we further identify the suspended animation problem with the existing
GNNs. Such a problem happens when the model depth reaches the suspended
animation limit, and the model will not respond to the training data any more
and become not learnable. Analysis about the causes of the suspended animation
problem with existing GNNs will be provided in this paper, whereas several
other peripheral factors that will impact the problem will be reported as well.
To resolve the problem, we introduce the GResNet (Graph Residual Network)
framework in this paper, which creates extensively connected highways to
involve nodes' raw features or intermediate representations throughout the
graph for all the model layers. Different from the other learning settings, the
extensive connections in the graph data will render the existing simple
residual learning methods fail to work. We prove the effectiveness of the
introduced new graph residual terms from the norm preservation perspective,
which will help avoid dramatic changes to the node's representations between
sequential layers. Detailed studies about the GResNet framework for many
existing GNNs, including GCN, GAT and LoopyNet, will be reported in the paper
with extensive empirical experiments on real-world benchmark datasets.
