DSTP-RNN: a dual-stage two-phase attention-based recurrent neural
  networks for long-term and multivariate time series prediction
  Long-term prediction of multivariate time series is still an important but
challenging problem. The key to solve this problem is to capture the spatial
correlations at the same time, the spatio-temporal relationships at different
times and the long-term dependence of the temporal relationships between
different series. Attention-based recurrent neural networks (RNN) can
effectively represent the dynamic spatio-temporal relationships between
exogenous series and target series, but it only performs well in one-step time
prediction and short-term time prediction. In this paper, inspired by human
attention mechanism including the dual-stage two-phase (DSTP) model and the
influence mechanism of target information and non-target information, we
propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-2 respectively for long-term
time series prediction. Specifically, we first propose the DSTP-based structure
to enhance the spatial correlations between exogenous series. The first phase
produces violent but decentralized response weight, while the second phase
leads to stationary and concentrated response weight. Secondly, we employ
multiple attentions on target series to boost the long-term dependence.
Finally, we study the performance of deep spatial attention mechanism and
provide experiment and interpretation. Our methods outperform nine baseline
methods on four datasets in the fields of energy, finance, environment and
medicine, respectively.
