Simplicial Attention Networks
  Graph representation learning methods have mostly been limited to the
modelling of node-wise interactions. Recently, there has been an increased
interest in understanding how higher-order structures can be utilised to
further enhance the learning abilities of graph neural networks (GNNs) in
combinatorial spaces. Simplicial Neural Networks (SNNs) naturally model these
interactions by performing message passing on simplicial complexes,
higher-dimensional generalisations of graphs. Nonetheless, the computations
performed by most existent SNNs are strictly tied to the combinatorial
structure of the complex. Leveraging the success of attention mechanisms in
structured domains, we propose Simplicial Attention Networks (SAT), a new type
of simplicial network that dynamically weighs the interactions between
neighbouring simplicies and can readily adapt to novel structures.
Additionally, we propose a signed attention mechanism that makes SAT
orientation equivariant, a desirable property for models operating on (co)chain
complexes. We demonstrate that SAT outperforms existent convolutional SNNs and
GNNs in two image and trajectory classification tasks.
