Knot Selection in Sparse Gaussian Processes
  Knot-based, sparse Gaussian processes have enjoyed considerable success as
scalable approximations to full Gaussian processes. Problems can occur,
however, when knot selection is done by optimizing the marginal likelihood. For
example, the marginal likelihood surface is highly multimodal, which can cause
suboptimal knot placement where some knots serve practically no function. This
is especially a problem when many more knots are used than are necessary,
resulting in extra computational cost for little to no gains in accuracy.
  We propose a one-at-a-time knot selection algorithm to select both the number
and placement of knots. Our algorithm uses Bayesian optimization to efficiently
propose knots that are likely to be good and largely avoids the pathologies
encountered when using the marginal likelihood as the objective function. We
provide empirical results showing improved accuracy and speed over the current
standard approaches.
