Ensemble Multi-task Gaussian Process Regression with Multiple Latent
  Processes
  Multi-task/Multi-output learning seeks to exploit correlation among tasks to
enhance performance over learning or solving each task independently. In this
paper, we investigate this problem in the context of Gaussian Processes (GPs)
and propose a new model which learns a mixture of latent processes by
decomposing the covariance matrix into a sum of structured hidden components
each of which is controlled by a latent GP over input features and a "weight"
over tasks. From this sum structure, we propose a parallelizable parameter
learning algorithm with a predetermined initialization for the "weights". We
also notice that an ensemble parameter learning approach using mini-batches of
training data not only reduces the computation complexity of learning but also
improves the regression performance. We evaluate our model on two datasets, the
smaller Swiss Jura dataset and another relatively larger ATMS dataset from
NOAA. Substantial improvements are observed compared with established
alternatives.
