Overcoming Mean-Field Approximations in Recurrent Gaussian Process
  Models
  We identify a new variational inference scheme for dynamical systems whose
transition function is modelled by a Gaussian process. Inference in this
setting has either employed computationally intensive MCMC methods, or relied
on factorisations of the variational posterior. As we demonstrate in our
experiments, the factorisation between latent system states and transition
function can lead to a miscalibrated posterior and to learning unnecessarily
large noise terms. We eliminate this factorisation by explicitly modelling the
dependence between state trajectories and the Gaussian process posterior.
Samples of the latent states can then be tractably generated by conditioning on
this representation. The method we obtain (VCDT: variationally coupled dynamics
and trajectories) gives better predictive performance and more calibrated
estimates of the transition function, yet maintains the same time and space
complexities as mean-field methods. Code is available at:
github.com/ialong/GPt.
