The interplay between system identification and machine learning
  Learning from examples is one of the key problems in science and engineering.
It deals with function reconstruction from a finite set of direct and noisy
samples. Regularization in reproducing kernel Hilbert spaces (RKHSs) is widely
used to solve this task and includes powerful estimators such as regularization
networks. Recent achievements include the proof of the statistical consistency
of these kernel- based approaches. Parallel to this, many different system
identification techniques have been developed but the interaction with machine
learning does not appear so strong yet. One reason is that the RKHSs usually
employed in machine learning do not embed the information available on dynamic
systems, e.g. BIBO stability. In addition, in system identification the
independent data assumptions routinely adopted in machine learning are never
satisfied in practice. This paper provides new results which strengthen the
connection between system identification and machine learning. Our starting
point is the introduction of RKHSs of dynamic systems. They contain functionals
over spaces defined by system inputs and allow to interpret system
identification as learning from examples. In both linear and nonlinear
settings, it is shown that this perspective permits to derive in a relatively
simple way conditions on RKHS stability (i.e. the property of containing only
BIBO stable systems or predictors), also facilitating the design of new kernels
for system identification. Furthermore, we prove the convergence of the
regularized estimator to the optimal predictor under conditions typical of
dynamic systems.
