Towards Deeper Graph Neural Networks
  Graph neural networks have shown significant success in the field of graph
representation learning. Graph convolutions perform neighborhood aggregation
and represent one of the most important graph operations. Nevertheless, one
layer of these neighborhood aggregation methods only consider immediate
neighbors, and the performance decreases when going deeper to enable larger
receptive fields. Several recent studies attribute this performance
deterioration to the over-smoothing issue, which states that repeated
propagation makes node representations of different classes indistinguishable.
In this work, we study this observation systematically and develop new insights
towards deeper graph neural networks. First, we provide a systematical analysis
on this issue and argue that the key factor compromising the performance
significantly is the entanglement of representation transformation and
propagation in current graph convolution operations. After decoupling these two
operations, deeper graph neural networks can be used to learn graph node
representations from larger receptive fields. We further provide a theoretical
analysis of the above observation when building very deep models, which can
serve as a rigorous and gentle description of the over-smoothing issue. Based
on our theoretical and empirical analysis, we propose Deep Adaptive Graph
Neural Network (DAGNN) to adaptively incorporate information from large
receptive fields. A set of experiments on citation, co-authorship, and
co-purchase datasets have confirmed our analysis and insights and demonstrated
the superiority of our proposed methods.
