Spectral Graph Attention Network with Fast Eigen-approximation
  Variants of Graph Neural Networks (GNNs) for representation learning have
been proposed recently and achieved fruitful results in various fields. Among
them, Graph Attention Network (GAT) first employs a self-attention strategy to
learn attention weights for each edge in the spatial domain. However, learning
the attentions over edges can only focus on the local information of graphs and
greatly increases the computational costs. In this paper, we first introduce
the attention mechanism in the spectral domain of graphs and present Spectral
Graph Attention Network (SpGAT) that learns representations for different
frequency components regarding weighted filters and graph wavelets bases. In
this way, SpGAT can better capture global patterns of graphs in an efficient
manner with much fewer learned parameters than that of GAT. Further, to reduce
the computational cost of SpGAT brought by the eigen-decomposition, we propose
a fast approximation variant SpGAT-Cheby. We thoroughly evaluate the
performance of SpGAT and SpGAT-Cheby in semi-supervised node classification
tasks and verify the effectiveness of the learned attentions in the spectral
domain.
