Deep Learning with Kernel Flow Regularization for Time Series
  Forecasting
  Long Short-Term Memory (LSTM) neural networks have been widely used for time
series forecasting problems. However, LSTMs are prone to overfitting and
performance reduction during test phases. Several different regularization
techniques have been shown in literature to prevent overfitting problems in
neural networks. In this paper, first, we introduce application of kernel flow
methods for time series forecasting in general. Afterward, we examine the
effectiveness of applying kernel flow regularization on LSTM layers to avoid
overfitting problems. We describe a regularization method by applying kernel
flow loss function on LSTM layers. In experimental results, we show that kernel
flow outperforms baseline models on time series forecasting benchmarks. We also
compare the effect of dropout and kernel flow regularization techniques on
LSTMs. The experimental results illustrate that kernel flow achieves similar
regularization effect to dropout. It also shows that the best results is
obtained using both kernel flow and dropout regularizations with early stopping
on LSTM layers on some time series datasets (e.g. power-load demand forecasts).
