Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time
  Series Prediction
  As industrial systems become more complex and monitoring sensors for
everything from surveillance to our health become more ubiquitous, multivariate
time series prediction is taking an important place in the smooth-running of
our society. A recurrent neural network with attention to help extend the
prediction windows is the current-state-of-the-art for this task. However, we
argue that their vanishing gradients, short memories, and serial architecture
make RNNs fundamentally unsuited to long-horizon forecasting with complex data.
Temporal convolutional networks (TCNs) do not suffer from gradient problems and
they support parallel calculations, making them a more appropriate choice.
Additionally, they have longer memories than RNNs, albeit with some instability
and efficiency problems. Hence, we propose a framework, called PSTA-TCN, that
combines a parallel spatio-temporal attention mechanism to extract dynamic
internal correlations with stacked TCN backbones to extract features from
different window sizes. The framework makes full use parallel calculations to
dramatically reduce training times, while substantially increasing accuracy
with stable prediction windows up to 13 times longer than the status quo.
