How to Find Your Friendly Neighborhood: Graph Attention Design with
  Self-Supervision
  Attention mechanism in graph neural networks is designed to assign larger
weights to important neighbor nodes for better representation. However, what
graph attention learns is not understood well, particularly when graphs are
noisy. In this paper, we propose a self-supervised graph attention network
(SuperGAT), an improved graph attention model for noisy graphs. Specifically,
we exploit two attention forms compatible with a self-supervised task to
predict edges, whose presence and absence contain the inherent information
about the importance of the relationships between nodes. By encoding edges,
SuperGAT learns more expressive attention in distinguishing mislinked
neighbors. We find two graph characteristics influence the effectiveness of
attention forms and self-supervision: homophily and average degree. Thus, our
recipe provides guidance on which attention design to use when those two graph
characteristics are known. Our experiment on 17 real-world datasets
demonstrates that our recipe generalizes across 15 datasets of them, and our
models designed by recipe show improved performance over baselines.
