Autoformer: Decomposition Transformers with Auto-Correlation for
  Long-Term Series Forecasting
  Extending the forecasting time is a critical demand for real applications,
such as extreme weather early warning and long-term energy consumption
planning. This paper studies the long-term forecasting problem of time series.
Prior Transformer-based models adopt various self-attention mechanisms to
discover the long-range dependencies. However, intricate temporal patterns of
the long-term future prohibit the model from finding reliable dependencies.
Also, Transformers have to adopt the sparse versions of point-wise
self-attentions for long series efficiency, resulting in the information
utilization bottleneck. Going beyond Transformers, we design Autoformer as a
novel decomposition architecture with an Auto-Correlation mechanism. We break
with the pre-processing convention of series decomposition and renovate it as a
basic inner block of deep models. This design empowers Autoformer with
progressive decomposition capacities for complex time series. Further, inspired
by the stochastic process theory, we design the Auto-Correlation mechanism
based on the series periodicity, which conducts the dependencies discovery and
representation aggregation at the sub-series level. Auto-Correlation
outperforms self-attention in both efficiency and accuracy. In long-term
forecasting, Autoformer yields state-of-the-art accuracy, with a 38% relative
improvement on six benchmarks, covering five practical applications: energy,
traffic, economics, weather and disease. Code is available at this repository:
\url{https://github.com/thuml/Autoformer}.
