A Note on Over-Smoothing for Graph Neural Networks
  Graph Neural Networks (GNNs) have achieved a lot of success on
graph-structured data. However, it is observed that the performance of graph
neural networks does not improve as the number of layers increases. This
effect, known as over-smoothing, has been analyzed mostly in linear cases. In
this paper, we build upon previous results \cite{oono2019graph} to further
analyze the over-smoothing effect in the general graph neural network
architecture. We show when the weight matrix satisfies the conditions
determined by the spectrum of augmented normalized Laplacian, the Dirichlet
energy of embeddings will converge to zero, resulting in the loss of
discriminative power. Using Dirichlet energy to measure "expressiveness" of
embedding is conceptually clean; it leads to simpler proofs than
\cite{oono2019graph} and can handle more non-linearities.
