Anomaly Transformer: Time Series Anomaly Detection with Association
  Discrepancy
  Unsupervised detection of anomaly points in time series is a challenging
problem, which requires the model to derive a distinguishable criterion.
Previous methods tackle the problem mainly through learning pointwise
representation or pairwise association, however, neither is sufficient to
reason about the intricate dynamics. Recently, Transformers have shown great
power in unified modeling of pointwise representation and pairwise association,
and we find that the self-attention weight distribution of each time point can
embody rich association with the whole series. Our key observation is that due
to the rarity of anomalies, it is extremely difficult to build nontrivial
associations from abnormal points to the whole series, thereby, the anomalies'
associations shall mainly concentrate on their adjacent time points. This
adjacent-concentration bias implies an association-based criterion inherently
distinguishable between normal and abnormal points, which we highlight through
the \emph{Association Discrepancy}. Technically, we propose the \emph{Anomaly
Transformer} with a new \emph{Anomaly-Attention} mechanism to compute the
association discrepancy. A minimax strategy is devised to amplify the
normal-abnormal distinguishability of the association discrepancy. The Anomaly
Transformer achieves state-of-the-art results on six unsupervised time series
anomaly detection benchmarks of three applications: service monitoring, space &
earth exploration, and water treatment.
