A Similarity Measure of Gaussian Process Predictive Distributions
  Some scenarios require the computation of a predictive distribution of a new
value evaluated on an objective function conditioned on previous observations.
We are interested on using a model that makes valid assumptions on the
objective function whose values we are trying to predict. Some of these
assumptions may be smoothness or stationarity. Gaussian process (GPs) are
probabilistic models that can be interpreted as flexible distributions over
functions. They encode the assumptions through covariance functions, making
hypotheses about new data through a predictive distribution by being fitted to
old observations. We can face the case where several GPs are used to model
different objective functions. GPs are non-parametric models whose complexity
is cubic on the number of observations. A measure that represents how similar
is one GP predictive distribution with respect to another would be useful to
stop using one GP when they are modelling functions of the same input space. We
are really inferring that two objective functions are correlated, so one GP is
enough to model both of them by performing a transformation of the prediction
of the other function in case of inverse correlation. We show empirical
evidence in a set of synthetic and benchmark experiments that GPs predictive
distributions can be compared and that one of them is enough to predict two
correlated functions in the same input space. This similarity metric could be
extremely useful used to discard objectives in Bayesian many-objective
optimization.
