Learning Attributed Graph Representations with Communicative Message
  Passing Transformer
  Constructing appropriate representations of molecules lies at the core of
numerous tasks such as material science, chemistry and drug designs. Recent
researches abstract molecules as attributed graphs and employ graph neural
networks (GNN) for molecular representation learning, which have made
remarkable achievements in molecular graph modeling. Albeit powerful, current
models either are based on local aggregation operations and thus miss
higher-order graph properties or focus on only node information without fully
using the edge information. For this sake, we propose a Communicative Message
Passing Transformer (CoMPT) neural network to improve the molecular graph
representation by reinforcing message interactions between nodes and edges
based on the Transformer architecture. Unlike the previous transformer-style
GNNs that treat molecules as fully connected graphs, we introduce a message
diffusion mechanism to leverage the graph connectivity inductive bias and
reduce the message enrichment explosion. Extensive experiments demonstrated
that the proposed model obtained superior performances (around 4$\%$ on
average) against state-of-the-art baselines on seven chemical property datasets
(graph-level tasks) and two chemical shift datasets (node-level tasks). Further
visualization studies also indicated a better representation capacity achieved
by our model.
