Deep Learning for Explicitly Modeling Optimization Landscapes
  In all but the most trivial optimization problems, the structure of the
solutions exhibit complex interdependencies between the input parameters.
Decades of research with stochastic search techniques has shown the benefit of
explicitly modeling the interactions between sets of parameters and the overall
quality of the solutions discovered. We demonstrate a novel method, based on
learning deep networks, to model the global landscapes of optimization
problems. To represent the search space concisely and accurately, the deep
networks must encode information about the underlying parameter interactions
and their contributions to the quality of the solution. Once the networks are
trained, the networks are probed to reveal parameter combinations with high
expected performance with respect to the optimization task. These estimates are
used to initialize fast, randomized, local search algorithms, which in turn
expose more information about the search space that is subsequently used to
refine the models. We demonstrate the technique on multiple optimization
problems that have arisen in a variety of real-world domains, including:
packing, graphics, job scheduling, layout and compression. The problems include
combinatoric search spaces, discontinuous and highly non-linear spaces, and
span binary, higher-cardinality discrete, as well as continuous parameters.
Strengths, limitations, and extensions of the approach are extensively
discussed and demonstrated.
