Practical Bayesian Optimization of Objectives with Conditioning
  Variables
  Bayesian optimization is a class of data efficient model based algorithms
typically focused on global optimization. We consider the more general case
where a user is faced with multiple problems that each need to be optimized
conditional on a state variable, for example given a range of cities with
different patient distributions, we optimize the ambulance locations
conditioned on patient distribution. Given partitions of CIFAR-10, we optimize
CNN hyperparameters for each partition. Similarity across objectives boosts
optimization of each objective in two ways: in modelling by data sharing across
objectives, and also in acquisition by quantifying how a single point on one
objective can provide benefit to all objectives. For this we propose a
framework for conditional optimization: ConBO. This can be built on top of a
range of acquisition functions and we propose a new Hybrid Knowledge Gradient
acquisition function. The resulting method is intuitive and theoretically
grounded, performs either similar to or significantly better than recently
published works on a range of problems, and is easily parallelized to collect a
batch of points.
