Dimensionality Reduction Meets Message Passing for Graph Node Embeddings
  Graph Neural Networks (GNNs) have become a popular approach for various
applications, ranging from social network analysis to modeling chemical
properties of molecules. While GNNs often show remarkable performance on public
datasets, they can struggle to learn long-range dependencies in the data due to
over-smoothing and over-squashing tendencies. To alleviate this challenge, we
propose PCAPass, a method which combines Principal Component Analysis (PCA) and
message passing for generating node embeddings in an unsupervised manner and
leverages gradient boosted decision trees for classification tasks. We show
empirically that this approach provides competitive performance compared to
popular GNNs on node classification benchmarks, while gathering information
from longer distance neighborhoods. Our research demonstrates that applying
dimensionality reduction with message passing and skip connections is a
promising mechanism for aggregating long-range dependencies in graph structured
data.
