Multi-Level Attention Pooling for Graph Neural Networks: Unifying Graph
  Representations with Multiple Localities
  Graph neural networks (GNNs) have been widely used to learn vector
representation of graph-structured data and achieved better task performance
than conventional methods. The foundation of GNNs is the message passing
procedure, which propagates the information in a node to its neighbors. Since
this procedure proceeds one step per layer, the range of the information
propagation among nodes is small in the lower layers, and it expands toward the
higher layers. Therefore, a GNN model has to be deep enough to capture global
structural information in a graph. On the other hand, it is known that deep GNN
models suffer from performance degradation because they lose nodes' local
information, which would be essential for good model performance, through many
message passing steps. In this study, we propose multi-level attention pooling
(MLAP) for graph-level classification tasks, which can adapt to both local and
global structural information in a graph. It has an attention pooling layer for
each message passing step and computes the final graph representation by
unifying the layer-wise graph representations. The MLAP architecture allows
models to utilize the structural information of graphs with multiple levels of
localities because it preserves layer-wise information before losing them due
to oversmoothing. Results of our experiments show that the MLAP architecture
improves the graph classification performance compared to the baseline
architectures. In addition, analyses on the layer-wise graph representations
suggest that aggregating information from multiple levels of localities indeed
has the potential to improve the discriminability of learned graph
representations.
