Bayesian Optimization with Binary Auxiliary Information
  This paper presents novel mixed-type Bayesian optimization (BO) algorithms to
accelerate the optimization of a target objective function by exploiting
correlated auxiliary information of binary type that can be more cheaply
obtained, such as in policy search for reinforcement learning and
hyperparameter tuning of machine learning models with early stopping. To
achieve this, we first propose a mixed-type multi-output Gaussian process
(MOGP) to jointly model the continuous target function and binary auxiliary
functions. Then, we propose information-based acquisition functions such as
mixed-type entropy search (MT-ES) and mixed-type predictive ES (MT-PES) for
mixed-type BO based on the MOGP predictive belief of the target and auxiliary
functions. The exact acquisition functions of MT-ES and MT-PES cannot be
computed in closed form and need to be approximated. We derive an efficient
approximation of MT-PES via a novel mixed-type random features approximation of
the MOGP model whose cross-correlation structure between the target and
auxiliary functions can be exploited for improving the belief of the global
target maximizer using observations from evaluating these functions. We propose
new practical constraints to relate the global target maximizer to the binary
auxiliary functions. We empirically evaluate the performance of MT-ES and
MT-PES with synthetic and real-world experiments.
