Improving Hyperparameter Optimization by Planning Ahead
  Hyperparameter optimization (HPO) is generally treated as a bi-level
optimization problem that involves fitting a (probabilistic) surrogate model to
a set of observed hyperparameter responses, e.g. validation loss, and
consequently maximizing an acquisition function using a surrogate model to
identify good hyperparameter candidates for evaluation. The choice of a
surrogate and/or acquisition function can be further improved via knowledge
transfer across related tasks. In this paper, we propose a novel transfer
learning approach, defined within the context of model-based reinforcement
learning, where we represent the surrogate as an ensemble of probabilistic
models that allows trajectory sampling. We further propose a new variant of
model predictive control which employs a simple look-ahead strategy as a policy
that optimizes a sequence of actions, representing hyperparameter candidates to
expedite HPO. Our experiments on three meta-datasets comparing to
state-of-the-art HPO algorithms including a model-free reinforcement learning
approach show that the proposed method can outperform all baselines by
exploiting a simple planning-based policy.
