Model inference for Ordinary Differential Equations by parametric
  polynomial kernel regression
  Model inference for dynamical systems aims to estimate the future behaviour
of a system from observations. Purely model-free statistical methods, such as
Artificial Neural Networks, tend to perform poorly for such tasks. They are
therefore not well suited to many questions from applications, for example in
Bayesian filtering and reliability estimation.
  This work introduces a parametric polynomial kernel method that can be used
for inferring the future behaviour of Ordinary Differential Equation models,
including chaotic dynamical systems, from observations. Using numerical
integration techniques, parametric representations of Ordinary Differential
Equations can be learnt using Backpropagation and Stochastic Gradient Descent.
The polynomial technique presented here is based on a nonparametric method,
kernel ridge regression. However, the time complexity of nonparametric kernel
ridge regression scales cubically with the number of training data points. Our
parametric polynomial method avoids this manifestation of the curse of
dimensionality, which becomes particularly relevant when working with large
time series data sets.
  Two numerical demonstrations are presented. First, a simple regression test
case is used to illustrate the method and to compare the performance with
standard Artificial Neural Network techniques. Second, a more substantial test
case is the inference of a chaotic spatio-temporal dynamical system, the
Lorenz--Emanuel system, from observations. Our method was able to successfully
track the future behaviour of the system over time periods much larger than the
training data sampling rate. Finally, some limitations of the method are
presented, as well as proposed directions for future work to mitigate these
limitations.
