Efficient Cross-Validation of Echo State Networks
  Echo State Networks (ESNs) are known for their fast and precise one-shot
learning of time series. But they often need good hyper-parameter tuning for
best performance. For this good validation is key, but usually, a single
validation split is used. In this rather practical contribution we suggest
several schemes for cross-validating ESNs and introduce an efficient algorithm
for implementing them. The component that dominates the time complexity of the
already quite fast ESN training remains constant (does not scale up with $k$)
in our proposed method of doing $k$-fold cross-validation. The component that
does scale linearly with $k$ starts dominating only in some not very common
situations. Thus in many situations $k$-fold cross-validation of ESNs can be
done for virtually the same time complexity as a simple single split
validation. Space complexity can also remain the same. We also discuss when the
proposed validation schemes for ESNs could be beneficial and empirically
investigate them on several different real-world datasets.
