Learning Recurrent Neural Net Models of Nonlinear Systems
  We consider the following learning problem: Given sample pairs of input and
output signals generated by an unknown nonlinear system (which is not assumed
to be causal or time-invariant), we wish to find a continuous-time recurrent
neural net with hyperbolic tangent activation function that approximately
reproduces the underlying i/o behavior with high confidence. Leveraging earlier
work concerned with matching output derivatives up to a given finite order, we
reformulate the learning problem in familiar system-theoretic language and
derive quantitative guarantees on the sup-norm risk of the learned model in
terms of the number of neurons, the sample size, the number of derivatives
being matched, and the regularity properties of the inputs, the outputs, and
the unknown i/o map.
