Liquid Time-constant Recurrent Neural Networks as Universal
  Approximators
  In this paper, we introduce the notion of liquid time-constant (LTC)
recurrent neural networks (RNN)s, a subclass of continuous-time RNNs, with
varying neuronal time-constant realized by their nonlinear synaptic
transmission model. This feature is inspired by the communication principles in
the nervous system of small species. It enables the model to approximate
continuous mapping with a small number of computational units. We show that any
finite trajectory of an $n$-dimensional continuous dynamical system can be
approximated by the internal state of the hidden units and $n$ output units of
an LTC network. Here, we also theoretically find bounds on their neuronal
states and varying time-constant.
