MQTransformer: Multi-Horizon Forecasts with Context Dependent and
  Feedback-Aware Attention
  Recent advances in neural forecasting have produced major improvements in
accuracy for probabilistic demand prediction. In this work, we propose novel
improvements to the current state of the art by incorporating changes inspired
by recent advances in Transformer architectures for Natural Language
Processing. We develop a novel decoder-encoder attention for context-alignment,
improving forecasting accuracy by allowing the network to study its own history
based on the context for which it is producing a forecast. We also present a
novel positional encoding that allows the neural network to learn
context-dependent seasonality functions as well as arbitrary holiday distances.
Finally we show that the current state of the art MQ-Forecaster (Wen et al.,
2017) models display excess variability by failing to leverage previous errors
in the forecast to improve accuracy. We propose a novel decoder-self attention
scheme for forecasting that produces significant improvements in the excess
variation of the forecast.
