An Empirical Study of the L2-Boost technique with Echo State Networks
  A particular case of Recurrent Neural Network (RNN) was introduced at the
beginning of the 2000s under the name of Echo State Networks (ESNs). The ESN
model overcomes the limitations during the training of the RNNs while
introducing no significant disadvantages. Although the model presents some
well-identified drawbacks when the parameters are not well initialised. The
performance of an ESN is highly dependent on its internal parameters and
pattern of connectivity of the hidden-hidden weights Often, the tuning of the
network parameters can be hard and can impact in the accuracy of the models.
  In this work, we investigate the performance of a specific boosting technique
(called L2-Boost) with ESNs as single predictors. The L2-Boost technique has
been shown to be an effective tool to combine "weak" predictors in regression
problems. In this study, we use an ensemble of random initialized ESNs (without
control their parameters) as "weak" predictors of the boosting procedure. We
evaluate our approach on five well-know time-series benchmark problems.
Additionally, we compare this technique with a baseline approach that consists
of averaging the prediction of an ensemble of ESNs.
