ES-dRNN with Dynamic Attention for Short-Term Load Forecasting
  Short-term load forecasting (STLF) is a challenging problem due to the
complex nature of the time series expressing multiple seasonality and varying
variance. This paper proposes an extension of a hybrid forecasting model
combining exponential smoothing and dilated recurrent neural network (ES-dRNN)
with a mechanism for dynamic attention. We propose a new gated recurrent cell
-- attentive dilated recurrent cell, which implements an attention mechanism
for dynamic weighting of input vector components. The most relevant components
are assigned greater weights, which are subsequently dynamically fine-tuned.
This attention mechanism helps the model to select input information and, along
with other mechanisms implemented in ES-dRNN, such as adaptive time series
processing, cross-learning, and multiple dilation, leads to a significant
improvement in accuracy when compared to well-established statistical and
state-of-the-art machine learning forecasting models. This was confirmed in the
extensive experimental study concerning STLF for 35 European countries.
