From Canonical Correlation Analysis to Self-supervised Graph Neural
  Networks
  We introduce a conceptually simple yet effective model for self-supervised
representation learning with graph data. It follows the previous methods that
generate two views of an input graph through data augmentation. However, unlike
contrastive methods that focus on instance-level discrimination, we optimize an
innovative feature-level objective inspired by classical Canonical Correlation
Analysis. Compared with other works, our approach requires none of the
parameterized mutual information estimator, additional projector, asymmetric
structures, and most importantly, negative samples which can be costly. We show
that the new objective essentially 1) aims at discarding augmentation-variant
information by learning invariant representations, and 2) can prevent
degenerated solutions by decorrelating features in different dimensions. Our
theoretical analysis further provides an understanding for the new objective
which can be equivalently seen as an instantiation of the Information
Bottleneck Principle under the self-supervised setting. Despite its simplicity,
our method performs competitively on seven public graph datasets. The code is
available at: https://github.com/hengruizhang98/CCA-SSG.
