Deep Echo State Networks with Uncertainty Quantification for
  Spatio-Temporal Forecasting
  Long-lead forecasting for spatio-temporal systems can often entail complex
nonlinear dynamics that are difficult to specify it a priori. Current
statistical methodologies for modeling these processes are often highly
parameterized and thus, challenging to implement from a computational
perspective. One potential parsimonious solution to this problem is a method
from the dynamical systems and engineering literature referred to as an echo
state network (ESN). ESN models use so-called {\it reservoir computing} to
efficiently compute recurrent neural network (RNN) forecasts. Moreover,
so-called "deep" models have recently been shown to be successful at predicting
high-dimensional complex nonlinear processes, particularly those with multiple
spatial and temporal scales of variability (such as we often find in
spatio-temporal environmental data). Here we introduce a deep ensemble ESN
(D-EESN) model. We present two versions of this model for spatio-temporal
processes that both produce forecasts and associated measures of uncertainty.
The first approach utilizes a bootstrap ensemble framework and the second is
developed within a hierarchical Bayesian framework (BD-EESN). This more general
hierarchical Bayesian framework naturally accommodates non-Gaussian data types
and multiple levels of uncertainties. The methodology is first applied to a
data set simulated from a novel non-Gaussian multiscale Lorenz-96 dynamical
system simulation model and then to a long-lead United States (U.S.) soil
moisture forecasting application.
