Geometry-aware Transformer for molecular property prediction
  Recently, graph neural networks (GNNs) have achieved remarkable performances
for quantum mechanical problems. However, a graph convolution can only cover a
localized region, and cannot capture long-range interactions of atoms. This
behavior is contrary to theoretical interatomic potentials, which is a
fundamental limitation of the spatial based GNNs. In this work, we propose a
novel attention-based framework for molecular property prediction tasks. We
represent a molecular conformation as a discrete atomic sequence combined by
atom-atom distance attributes, named Geometry-aware Transformer (GeoT). In
particular, we adopt a Transformer architecture, which has been widely used for
sequential data. Our proposed model trains sequential representations of
molecular graphs based on globally constructed attentions, maintaining all
spatial arrangements of atom pairs. Our method does not suffer from cost
intensive computations, such as angle calculations. The experimental results on
several public benchmarks and visualization maps verified that keeping the
long-range interatomic attributes can significantly improve the model
predictability.
