Healing Products of Gaussian Processes
  Gaussian processes (GPs) are nonparametric Bayesian models that have been
applied to regression and classification problems. One of the approaches to
alleviate their cubic training cost is the use of local GP experts trained on
subsets of the data. In particular, product-of-expert models combine the
predictive distributions of local experts through a tractable product
operation. While these expert models allow for massively distributed
computation, their predictions typically suffer from erratic behaviour of the
mean or uncalibrated uncertainty quantification. By calibrating predictions via
a tempered softmax weighting, we provide a solution to these problems for
multiple product-of-expert models, including the generalised product of experts
and the robust Bayesian committee machine. Furthermore, we leverage the optimal
transport literature and propose a new product-of-expert model that combines
predictions of local experts by computing their Wasserstein barycenter, which
can be applied to both regression and classification.
