End-to-End Learning of Deep Kernel Acquisition Functions for Bayesian
  Optimization
  For Bayesian optimization (BO) on high-dimensional data with complex
structure, neural network-based kernels for Gaussian processes (GPs) have been
used to learn flexible surrogate functions by the high representation power of
deep learning. However, existing methods train neural networks by maximizing
the marginal likelihood, which do not directly improve the BO performance. In
this paper, we propose a meta-learning method for BO with neural network-based
kernels that minimizes the expected gap between the true optimum value and the
best value found by BO. We model a policy, which takes the current evaluated
data points as input and outputs the next data point to be evaluated, by a
neural network, where neural network-based kernels, GPs, and mutual
information-based acquisition functions are used as its layers. With our model,
the neural network-based kernel is trained to be appropriate for the
acquisition function by backpropagating the gap through the acquisition
function and GP. Our model is trained by a reinforcement learning framework
from multiple tasks. Since the neural network is shared across different tasks,
we can gather knowledge on BO from multiple training tasks, and use the
knowledge for unseen test tasks. In experiments using three text document
datasets, we demonstrate that the proposed method achieves better BO
performance than the existing methods.
