Why Propagate Alone? Parallel Use of Labels and Features on Graphs
  Graph neural networks (GNNs) and label propagation represent two interrelated
modeling strategies designed to exploit graph structure in tasks such as node
property prediction. The former is typically based on stacked message-passing
layers that share neighborhood information to transform node features into
predictive embeddings. In contrast, the latter involves spreading label
information to unlabeled nodes via a parameter-free diffusion process, but
operates independently of the node features. Given then that the material
difference is merely whether features or labels are smoothed across the graph,
it is natural to consider combinations of the two for improving performance. In
this regard, it has recently been proposed to use a randomly-selected portion
of the training labels as GNN inputs, concatenated with the original node
features for making predictions on the remaining labels. This so-called label
trick accommodates the parallel use of features and labels, and is foundational
to many of the top-ranking submissions on the Open Graph Benchmark (OGB)
leaderboard. And yet despite its wide-spread adoption, thus far there has been
little attempt to carefully unpack exactly what statistical properties the
label trick introduces into the training pipeline, intended or otherwise. To
this end, we prove that under certain simplifying assumptions, the stochastic
label trick can be reduced to an interpretable, deterministic training
objective composed of two factors. The first is a data-fitting term that
naturally resolves potential label leakage issues, while the second serves as a
regularization factor conditioned on graph structure that adapts to graph size
and connectivity. Later, we leverage this perspective to motivate a broader
range of label trick use cases, and provide experiments to verify the efficacy
of these extensions.
