Tighter Bounds on the Log Marginal Likelihood of Gaussian Process
  Regression Using Conjugate Gradients
  We propose a lower bound on the log marginal likelihood of Gaussian process
regression models that can be computed without matrix factorisation of the full
kernel matrix. We show that approximate maximum likelihood learning of model
parameters by maximising our lower bound retains many of the sparse variational
approach benefits while reducing the bias introduced into parameter learning.
The basis of our bound is a more careful analysis of the log-determinant term
appearing in the log marginal likelihood, as well as using the method of
conjugate gradients to derive tight lower bounds on the term involving a
quadratic form. Our approach is a step forward in unifying methods relying on
lower bound maximisation (e.g. variational methods) and iterative approaches
based on conjugate gradients for training Gaussian processes. In experiments,
we show improved predictive performance with our model for a comparable amount
of training time compared to other conjugate gradient based approaches.
