Gaussian Graphical Models as an Ensemble Method for Distributed Gaussian
  Processes
  Distributed Gaussian process (DGP) is a popular approach to scale GP to big
data which divides the training data into some subsets, performs local
inference for each partition, and aggregates the results to acquire global
prediction. To combine the local predictions, the conditional independence
assumption is used which basically means there is a perfect diversity between
the subsets. Although it keeps the aggregation tractable, it is often violated
in practice and generally yields poor results. In this paper, we propose a
novel approach for aggregating the Gaussian experts' predictions by Gaussian
graphical model (GGM) where the target aggregation is defined as an unobserved
latent variable and the local predictions are the observed variables. We first
estimate the joint distribution of latent and observed variables using the
Expectation-Maximization (EM) algorithm. The interaction between experts can be
encoded by the precision matrix of the joint distribution and the aggregated
predictions are obtained based on the property of conditional Gaussian
distribution. Using both synthetic and real datasets, our experimental
evaluations illustrate that our new method outperforms other state-of-the-art
DGP approaches.
