A Differential Attention Fusion Model Based on Transformer for Time
  Series Forecasting
  Time series forecasting is widely used in the fields of equipment life cycle
forecasting, weather forecasting, traffic flow forecasting, and other fields.
Recently, some scholars have tried to apply Transformer to time series
forecasting because of its powerful parallel training ability. However, the
existing Transformer methods do not pay enough attention to the small time
segments that play a decisive role in prediction, making it insensitive to
small changes that affect the trend of time series, and it is difficult to
effectively learn continuous time-dependent features. To solve this problem, we
propose a differential attention fusion model based on Transformer, which
designs the differential layer, neighbor attention, sliding fusion mechanism,
and residual layer on the basis of classical Transformer architecture.
Specifically, the differences of adjacent time points are extracted and focused
by difference and neighbor attention. The sliding fusion mechanism fuses
various features of each time point so that the data can participate in
encoding and decoding without losing important information. The residual layer
including convolution and LSTM further learns the dependence between time
points and enables our model to carry out deeper training. A large number of
experiments on three datasets show that the prediction results produced by our
method are favorably comparable to the state-of-the-art.
