Cluster-based Input Weight Initialization for Echo State Networks
  Echo State Networks (ESNs) are a special type of recurrent neural networks
(RNNs), in which the input and recurrent connections are traditionally
generated randomly, and only the output weights are trained. Despite the recent
success of ESNs in various tasks of audio, image and radar recognition, we
postulate that a purely random initialization is not the ideal way of
initializing ESNs. The aim of this work is to propose an unsupervised
initialization of the input connections using the $K$-Means algorithm on the
training data. We show that for a large variety of datasets this initialization
performs equivalently or superior than a randomly initialized ESN whilst
needing significantly less reservoir neurons. Furthermore, we discuss that this
approach provides the opportunity to estimate a suitable size of the reservoir
based on prior knowledge about the data.
