Scalable L\'evy Process Priors for Spectral Kernel Learning
  Gaussian processes are rich distributions over functions, with generalization
properties determined by a kernel function. When used for long-range
extrapolation, predictions are particularly sensitive to the choice of kernel
parameters. It is therefore critical to account for kernel uncertainty in our
predictive distributions. We propose a distribution over kernels formed by
modelling a spectral mixture density with a L\'evy process. The resulting
distribution has support for all stationary covariances--including the popular
RBF, periodic, and Mat\'ern kernels--combined with inductive biases which
enable automatic and data efficient learning, long-range extrapolation, and
state of the art predictive performance. The proposed model also presents an
approach to spectral regularization, as the L\'evy process introduces a
sparsity-inducing prior over mixture components, allowing automatic selection
over model order and pruning of extraneous components. We exploit the algebraic
structure of the proposed process for $\mathcal{O}(n)$ training and
$\mathcal{O}(1)$ predictions. We perform extrapolations having reasonable
uncertainty estimates on several benchmarks, show that the proposed model can
recover flexible ground truth covariances and that it is robust to errors in
initialization.
