Adjoint-aided inference of Gaussian process driven differential
  equations
  Linear systems occur throughout engineering and the sciences, most notably as
differential equations. In many cases the forcing function for the system is
unknown, and interest lies in using noisy observations of the system to infer
the forcing, as well as other unknown parameters. In differential equations,
the forcing function is an unknown function of the independent variables
(typically time and space), and can be modelled as a Gaussian process (GP). In
this paper we show how the adjoint of a linear system can be used to
efficiently infer forcing functions modelled as GPs, after using a truncated
basis expansion of the GP kernel. We show how exact conjugate Bayesian
inference for the truncated GP can be achieved, in many cases with
substantially lower computation than would be required using MCMC methods. We
demonstrate the approach on systems of both ordinary and partial differential
equations, and by testing on synthetic data, show that the basis expansion
approach approximates well the true forcing with a modest number of basis
vectors. Finally, we show how to infer point estimates for the non-linear model
parameters, such as the kernel length-scales, using Bayesian optimisation.
