Paying Attention to Astronomical Transients: Photometric Classification
  with the Time-Series Transformer
  Future surveys such as the Legacy Survey of Space and Time (LSST) of the Vera
C. Rubin Observatory will observe an order of magnitude more astrophysical
transient events than any previous survey before. With this deluge of
photometric data, it will be impossible for all such events to be classified by
humans alone. Recent efforts have sought to leverage machine learning methods
to tackle the challenge of astronomical transient classification, with ever
improving success. Transformers are a recently developed deep learning
architecture, first proposed for natural language processing, that have shown a
great deal of recent success. In this work we develop a new transformer
architecture, which uses multi-head self attention at its core, for general
multi-variate time-series data. Furthermore, the proposed time-series
transformer architecture supports the inclusion of an arbitrary number of
additional features, while also offering interpretability. We apply the
time-series transformer to the task of photometric classification, minimising
the reliance of expert domain knowledge for feature selection, while achieving
results comparable to state-of-the-art photometric classification methods. We
achieve a weighted logarithmic-loss of 0.507 on imbalanced data in a
representative setting using data from the Photometric LSST Astronomical
Time-Series Classification Challenge (PLAsTiCC). Moreover, we achieve a
micro-averaged receiver operating characteristic area under curve of 0.98 and
micro-averaged precision-recall area under curve of 0.87.
