Deep learning with transfer functions: new applications in system
  identification
  This paper presents a linear dynamical operator described in terms of a
rational transfer function, endowed with a well-defined and efficient
back-propagation behavior for automatic derivatives computation. The operator
enables end-to-end training of structured networks containing linear transfer
functions and other differentiable units {by} exploiting standard deep learning
software.
  Two relevant applications of the operator in system identification are
presented. The first one consists in the integration of {prediction error
methods} in deep learning. The dynamical operator is included as {the} last
layer of a neural network in order to obtain the optimal one-step-ahead
prediction error.
  The second one considers identification of general block-oriented models from
quantized data. These block-oriented models are constructed by combining linear
dynamical operators with static nonlinearities described as standard
feed-forward neural networks. A custom loss function corresponding to the
log-likelihood of quantized output observations is defined. For gradient-based
optimization, the derivatives of the log-likelihood are computed by applying
the back-propagation algorithm through the whole network. Two system
identification benchmarks are used to show the effectiveness of the proposed
methodologies.
