Variational Inference for Uncertainty on the Inputs of Gaussian Process
  Models
  The Gaussian process latent variable model (GP-LVM) provides a flexible
approach for non-linear dimensionality reduction that has been widely applied.
However, the current approach for training GP-LVMs is based on maximum
likelihood, where the latent projection variables are maximized over rather
than integrated out. In this paper we present a Bayesian method for training
GP-LVMs by introducing a non-standard variational inference framework that
allows to approximately integrate out the latent variables and subsequently
train a GP-LVM by maximizing an analytic lower bound on the exact marginal
likelihood. We apply this method for learning a GP-LVM from iid observations
and for learning non-linear dynamical systems where the observations are
temporally correlated. We show that a benefit of the variational Bayesian
procedure is its robustness to overfitting and its ability to automatically
select the dimensionality of the nonlinear latent space. The resulting
framework is generic, flexible and easy to extend for other purposes, such as
Gaussian process regression with uncertain inputs and semi-supervised Gaussian
processes. We demonstrate our method on synthetic data and standard machine
learning benchmarks, as well as challenging real world datasets, including high
resolution video data.
