Sheaf Neural Networks with Connection Laplacians
A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that
operates on a sheaf, an object that equips a graph with vector spaces over its
nodes and edges and linear maps between these spaces. SNNs have been shown to
have useful theoretical properties that help tackle issues arising from
heterophily and over-smoothing. One complication intrinsic to these models is
finding a good sheaf for the task to be solved. Previous works proposed two
diametrically opposed approaches: manually constructing the sheaf based on
domain knowledge and learning the sheaf end-to-end using gradient-based
methods. However, domain knowledge is often insufficient, while learning a
sheaf could lead to overfitting and significant computational overhead. In this
work, we propose a novel way of computing sheaves drawing inspiration from
Riemannian geometry: we leverage the manifold assumption to compute
manifold-and-graph-aware orthogonal maps, which optimally align the tangent
spaces of neighbouring data points. We show that this approach achieves
promising results with less computational overhead when compared to previous
SNN models. Overall, this work provides an interesting connection between
algebraic topology and differential geometry, and we hope that it will spark
future research in this direction.