Discrete-time signatures and randomness in reservoir computing
  A new explanation of geometric nature of the reservoir computing phenomenon
is presented. Reservoir computing is understood in the literature as the
possibility of approximating input/output systems with randomly chosen
recurrent neural systems and a trained linear readout layer. Light is shed on
this phenomenon by constructing what is called strongly universal reservoir
systems as random projections of a family of state-space systems that generate
Volterra series expansions. This procedure yields a state-affine reservoir
system with randomly generated coefficients in a dimension that is
logarithmically reduced with respect to the original system. This reservoir
system is able to approximate any element in the fading memory filters class
just by training a different linear readout for each different filter. Explicit
expressions for the probability distributions needed in the generation of the
projected reservoir system are stated and bounds for the committed
approximation error are provided.
