Additive Tree-Structured Covariance Function for Conditional Parameter
  Spaces in Bayesian Optimization
  Bayesian optimization (BO) is a sample-efficient global optimization
algorithm for black-box functions which are expensive to evaluate. Existing
literature on model based optimization in conditional parameter spaces are
usually built on trees. In this work, we generalize the additive assumption to
tree-structured functions and propose an additive tree-structured covariance
function, showing improved sample-efficiency, wider applicability and greater
flexibility. Furthermore, by incorporating the structure information of
parameter spaces and the additive assumption in the BO loop, we develop a
parallel algorithm to optimize the acquisition function and this optimization
can be performed in a low dimensional space. We demonstrate our method on an
optimization benchmark function, as well as on a neural network model
compression problem, and experimental results show our approach significantly
outperforms the current state of the art for conditional parameter optimization
including SMAC, TPE and Jenatton et al. (2017).
