Combining Latent Space and Structured Kernels for Bayesian Optimization
  over Combinatorial Spaces
  We consider the problem of optimizing combinatorial spaces (e.g., sequences,
trees, and graphs) using expensive black-box function evaluations. For example,
optimizing molecules for drug design using physical lab experiments. Bayesian
optimization (BO) is an efficient framework for solving such problems by
intelligently selecting the inputs with high utility guided by a learned
surrogate model. A recent BO approach for combinatorial spaces is through a
reduction to BO over continuous spaces by learning a latent representation of
structures using deep generative models (DGMs). The selected input from the
continuous space is decoded into a discrete structure for performing function
evaluation. However, the surrogate model over the latent space only uses the
information learned by the DGM, which may not have the desired inductive bias
to approximate the target black-box function. To overcome this drawback, this
paper proposes a principled approach referred as LADDER. The key idea is to
define a novel structure-coupled kernel that explicitly integrates the
structural information from decoded structures with the learned latent space
representation for better surrogate modeling. Our experiments on real-world
benchmarks show that LADDER significantly improves over the BO over latent
space method, and performs better or similar to state-of-the-art methods.
