Evaluating Gaussian Process Metamodels and Sequential Designs for Noisy
  Level Set Estimation
  We consider the problem of learning the level set for which a noisy black-box
function exceeds a given threshold. To efficiently reconstruct the level set,
we investigate Gaussian process (GP) metamodels. Our focus is on strongly
stochastic samplers, in particular with heavy-tailed simulation noise and low
signal-to-noise ratio. To guard against noise misspecification, we assess the
performance of three variants: (i) GPs with Student-$t$ observations; (ii)
Student-$t$ processes (TPs); and (iii) classification GPs modeling the sign of
the response. In conjunction with these metamodels, we analyze several
acquisition functions for guiding the sequential experimental designs,
extending existing stepwise uncertainty reduction criteria to the stochastic
contour-finding context. This also motivates our development of (approximate)
updating formulas to efficiently compute such acquisition functions. Our
schemes are benchmarked by using a variety of synthetic experiments in 1--6
dimensions. We also consider an application of level set estimation for
determining the optimal exercise policy of Bermudan options in finance.
