Detect the Interactions that Matter in Matter: Geometric Attention for
  Many-Body Systems
  Attention mechanisms are developing into a viable alternative to
convolutional layers as elementary building block of NNs. Their main advantage
is that they are not restricted to capture local dependencies in the input, but
can draw arbitrary connections. This unprecedented capability coincides with
the long-standing problem of modeling global atomic interactions in molecular
force fields and other many-body problems. In its original formulation,
however, attention is not applicable to the continuous domains in which the
atoms live. For this purpose we propose a variant to describe geometric
relations for arbitrary atomic configurations in Euclidean space that also
respects all relevant physical symmetries. We furthermore demonstrate, how the
successive application of our learned attention matrices effectively translates
the molecular geometry into a set of individual atomic contributions
on-the-fly.
