Transformers in Time Series: A Survey
  Transformers have achieved superior performances in many tasks in natural
language processing and computer vision, which also intrigues great interests
in the time series community. Among multiple advantages of transformers, the
ability to capture long-range dependencies and interactions is especially
attractive for time series modeling, leading to exciting progress in various
time series applications. In this paper, we systematically review transformer
schemes for time series modeling by highlighting their strengths as well as
limitations through a new taxonomy to summarize existing time series
transformers in two perspectives. From the perspective of network
modifications, we summarize the adaptations of module level and architecture
level of the time series transformers. From the perspective of applications, we
categorize time series transformers based on common tasks including
forecasting, anomaly detection, and classification. Empirically, we perform
robust analysis, model size analysis, and seasonal-trend decomposition analysis
to study how Transformers perform in time series. Finally, we discuss and
suggest future directions to provide useful research guidance. A corresponding
resource list that will be continuously updated can be found in the GitHub
repository. To the best of our knowledge, this paper is the first work to
comprehensively and systematically summarize the recent advances of
Transformers for modeling time series data. We hope this survey will ignite
further research interests in time series Transformers.
