Recursive Estimation for Sparse Gaussian Process Regression
  Gaussian Processes (GPs) are powerful kernelized methods for non-parameteric
regression used in many applications. However, their use is limited to a few
thousand of training samples due to their cubic time complexity. In order to
scale GPs to larger datasets, several sparse approximations based on so-called
inducing points have been proposed in the literature. In this work we
investigate the connection between a general class of sparse inducing point GP
regression methods and Bayesian recursive estimation which enables Kalman
Filter like updating for online learning. The majority of previous work has
focused on the batch setting, in particular for learning the model parameters
and the position of the inducing points, here instead we focus on training with
mini-batches. By exploiting the Kalman filter formulation, we propose a novel
approach that estimates such parameters by recursively propagating the
analytical gradients of the posterior over mini-batches of the data. Compared
to state of the art methods, our method keeps analytic updates for the mean and
covariance of the posterior, thus reducing drastically the size of the
optimization problem. We show that our method achieves faster convergence and
superior performance compared to state of the art sequential Gaussian Process
regression on synthetic GP as well as real-world data with up to a million of
data samples.
