Residual Gaussian Process: A Tractable Nonparametric Bayesian Emulator
  for Multi-fidelity Simulations
  Challenges in multi-fidelity modeling relate to accuracy, uncertainty
estimation and high-dimensionality. A novel additive structure is introduced in
which the highest fidelity solution is written as a sum of the lowest fidelity
solution and residuals between the solutions at successive fidelity levels,
with Gaussian process priors placed over the low fidelity solution and each of
the residuals. The resulting model is equipped with a closed-form solution for
the predictive posterior, making it applicable to advanced, high-dimensional
tasks that require uncertainty estimation. Its advantages are demonstrated on
univariate benchmarks and on three challenging multivariate problems. It is
shown how active learning can be used to enhance the model, especially with a
limited computational budget. Furthermore, error bounds are derived for the
mean prediction in the univariate case.
