Backpropagation Algorithms and Reservoir Computing in Recurrent Neural
  Networks for the Forecasting of Complex Spatiotemporal Dynamics
  We examine the efficiency of Recurrent Neural Networks in forecasting the
spatiotemporal dynamics of high dimensional and reduced order complex systems
using Reservoir Computing (RC) and Backpropagation through time (BPTT) for
gated network architectures. We highlight advantages and limitations of each
method and discuss their implementation for parallel computing architectures.
We quantify the relative prediction accuracy of these algorithms for the
longterm forecasting of chaotic systems using as benchmarks the Lorenz-96 and
the Kuramoto-Sivashinsky (KS) equations. We find that, when the full state
dynamics are available for training, RC outperforms BPTT approaches in terms of
predictive performance and in capturing of the long-term statistics, while at
the same time requiring much less training time. However, in the case of
reduced order data, large scale RC models can be unstable and more likely than
the BPTT algorithms to diverge. In contrast, RNNs trained via BPTT show
superior forecasting abilities and capture well the dynamics of reduced order
systems. Furthermore, the present study quantifies for the first time the
Lyapunov Spectrum of the KS equation with BPTT, achieving similar accuracy as
RC. This study establishes that RNNs are a potent computational framework for
the learning and forecasting of complex spatiotemporal systems.
