SkipNode: On Alleviating Over-smoothing for Deep Graph Convolutional
  Networks
  Over-smoothing is a challenging problem, which degrades the performance of
deep graph convolutional networks (GCNs). However, existing studies for
alleviating the over-smoothing problem lack either generality or effectiveness.
In this paper, we analyze the underlying issues behind the over-smoothing
problem, i.e., feature-diversity degeneration, gradient vanishing, and model
weights over-decaying. Inspired by this, we propose a simple yet effective
plug-and-play module, SkipNode, to alleviate over-smoothing. Specifically, for
each middle layer of a GCN model, SkipNode randomly (or based on node degree)
selects nodes to skip the convolutional operation by directly feeding their
input features to the nonlinear function. Analytically, 1) skipping the
convolutional operation prevents the features from losing diversity; and 2) the
"skipped" nodes enable gradients to be directly passed back, thus mitigating
the gradient vanishing and model weights over-decaying issues. To demonstrate
the superiority of SkipNode, we conduct extensive experiments on nine popular
datasets, including both homophilic and heterophilic graphs, with different
graph sizes on two typical tasks: node classification and link prediction.
Specifically, 1) SkipNode has strong generalizability of being applied to
various GCN-based models on different datasets and tasks; and 2) SkipNode
outperforms recent state-of-the-art anti-over-smoothing plug-and-play modules,
i.e., DropEdge and DropNode, in different settings. Code will be made publicly
available on GitHub.
