Parallel Architecture and Hyperparameter Search via Successive Halving
  and Classification
  We present a simple and powerful algorithm for parallel black box
optimization called Successive Halving and Classification (SHAC). The algorithm
operates in $K$ stages of parallel function evaluations and trains a cascade of
binary classifiers to iteratively cull the undesirable regions of the search
space. SHAC is easy to implement, requires no tuning of its own configuration
parameters, is invariant to the scale of the objective function and can be
built using any choice of binary classifier. We adopt tree-based classifiers
within SHAC and achieve competitive performance against several strong
baselines for optimizing synthetic functions, hyperparameters and
architectures.
