The Parallel Knowledge Gradient Method for Batch Bayesian Optimization
  In many applications of black-box optimization, one can evaluate multiple
points simultaneously, e.g. when evaluating the performances of several
different neural network architectures in a parallel computing environment. In
this paper, we develop a novel batch Bayesian optimization algorithm --- the
parallel knowledge gradient method. By construction, this method provides the
one-step Bayes-optimal batch of points to sample. We provide an efficient
strategy for computing this Bayes-optimal batch of points, and we demonstrate
that the parallel knowledge gradient method finds global optima significantly
faster than previous batch Bayesian optimization algorithms on both synthetic
test functions and when tuning hyperparameters of practical machine learning
algorithms, especially when function evaluations are noisy.
