Conditional Deep Gaussian Processes: multi-fidelity kernel learning
  Deep Gaussian Processes (DGPs) were proposed as an expressive Bayesian model
capable of a mathematically grounded estimation of uncertainty. The
expressivity of DPGs results from not only the compositional character but the
distribution propagation within the hierarchy. Recently, [1] pointed out that
the hierarchical structure of DGP well suited modeling the multi-fidelity
regression, in which one is provided sparse observations with high precision
and plenty of low fidelity observations. We propose the conditional DGP model
in which the latent GPs are directly supported by the fixed lower fidelity
data. Then the moment matching method in [2] is applied to approximate the
marginal prior of conditional DGP with a GP. The obtained effective kernels are
implicit functions of the lower-fidelity data, manifesting the expressivity
contributed by distribution propagation within the hierarchy. The
hyperparameters are learned via optimizing the approximate marginal likelihood.
Experiments with synthetic and high dimensional data show comparable
performance against other multi-fidelity regression methods, variational
inference, and multi-output GP. We conclude that, with the low fidelity data
and the hierarchical DGP structure, the effective kernel encodes the inductive
bias for true function allowing the compositional freedom discussed in [3,4].
