Knot Selection in Sparse Gaussian Processes with a Variational Objective
  Function
  Sparse, knot-based Gaussian processes have enjoyed considerable success as
scalable approximations to full Gaussian processes. Certain sparse models can
be derived through specific variational approximations to the true posterior,
and knots can be selected to minimize the Kullback-Leibler divergence between
the approximate and true posterior. While this has been a successful approach,
simultaneous optimization of knots can be slow due to the number of parameters
being optimized. Furthermore, there have been few proposed methods for
selecting the number of knots, and no experimental results exist in the
literature. We propose using a one-at-a-time knot selection algorithm based on
Bayesian optimization to select the number and locations of knots. We showcase
the competitive performance of this method relative to simultaneous
optimization of knots on three benchmark data sets, but at a fraction of the
computational cost.
