TCCT: Tightly-Coupled Convolutional Transformer on Time Series
  Forecasting
  Time series forecasting is essential for a wide range of real-world
applications. Recent studies have shown the superiority of Transformer in
dealing with such problems, especially long sequence time series input(LSTI)
and long sequence time series forecasting(LSTF) problems. To improve the
efficiency and enhance the locality of Transformer, these studies combine
Transformer with CNN in varying degrees. However, their combinations are
loosely-coupled and do not make full use of CNN. To address this issue, we
propose the concept of tightly-coupled convolutional Transformer(TCCT) and
three TCCT architectures which apply transformed CNN architectures into
Transformer: (1) CSPAttention: through fusing CSPNet with self-attention
mechanism, the computation cost of self-attention mechanism is reduced by 30%
and the memory usage is reduced by 50% while achieving equivalent or beyond
prediction accuracy. (2) Dilated causal convolution: this method is to modify
the distilling operation proposed by Informer through replacing canonical
convolutional layers with dilated causal convolutional layers to gain
exponentially receptive field growth. (3) Passthrough mechanism: the
application of passthrough mechanism to stack of self-attention blocks helps
Transformer-like models get more fine-grained information with negligible extra
computation costs. Our experiments on real-world datasets show that our TCCT
architectures could greatly improve the performance of existing state-of-art
Transformer models on time series forecasting with much lower computation and
memory costs, including canonical Transformer, LogTrans and Informer.
