Deep Learning for Bayesian Optimization of Scientific Problems with
  High-Dimensional Structure
  Bayesian optimization (BO) is a popular paradigm for global optimization of
expensive black-box functions, but there are many domains where the function is
not completely black-box. The data may have some known structure (e.g.
symmetries) and/or the data generation process can yield useful intermediate or
auxiliary information in addition to the value of the optimization objective.
However, surrogate models traditionally employed in BO, such as Gaussian
Processes (GPs), scale poorly with dataset size and do not easily accommodate
known structure or auxiliary information. Instead, we propose performing BO on
complex, structured problems by using deep learning models with uncertainty, a
class of scalable surrogate models that have the representation power and
flexibility to handle structured data and exploit auxiliary information. We
demonstrate BO on a number of realistic problems in physics and chemistry,
including topology optimization of photonic crystal materials using
convolutional neural networks, and chemical property optimization of molecules
using graph neural networks. On these complex tasks, we show that neural
networks often outperform GPs as surrogate models for BO in terms of both
sampling efficiency and computational cost.
