Graph Attention Networks with Positional Embeddings
  Graph Neural Networks (GNNs) are deep learning methods which provide the
current state of the art performance in node classification tasks. GNNs often
assume homophily -- neighboring nodes having similar features and labels--, and
therefore may not be at their full potential when dealing with non-homophilic
graphs. In this work, we focus on addressing this limitation and enable Graph
Attention Networks (GAT), a commonly used variant of GNNs, to explore the
structural information within each graph locality. Inspired by the positional
encoding in the Transformers, we propose a framework, termed Graph Attentional
Networks with Positional Embeddings (GAT-POS), to enhance GATs with positional
embeddings which capture structural and positional information of the nodes in
the graph. In this framework, the positional embeddings are learned by a model
predictive of the graph context, plugged into an enhanced GAT architecture,
which is able to leverage both the positional and content information of each
node. The model is trained jointly to optimize for the task of node
classification as well as the task of predicting graph context. Experimental
results show that GAT-POS reaches remarkable improvement compared to strong GNN
baselines and recent structural embedding enhanced GNNs on non-homophilic
graphs.
