Addressing Over-Smoothing in Graph Neural Networks via Deep Supervision
  Learning useful node and graph representations with graph neural networks
(GNNs) is a challenging task. It is known that deep GNNs suffer from
over-smoothing where, as the number of layers increases, node representations
become nearly indistinguishable and model performance on the downstream task
degrades significantly. To address this problem, we propose deeply-supervised
GNNs (DSGNNs), i.e., GNNs enhanced with deep supervision where representations
learned at all layers are used for training. We show empirically that DSGNNs
are resilient to over-smoothing and can outperform competitive benchmarks on
node and graph property prediction problems.
