Deep Optimisation: Solving Combinatorial Optimisation Problems using
  Deep Neural Networks
  Deep Optimisation (DO) combines evolutionary search with Deep Neural Networks
(DNNs) in a novel way - not for optimising a learning algorithm, but for
finding a solution to an optimisation problem. Deep learning has been
successfully applied to classification, regression, decision and generative
tasks and in this paper we extend its application to solving optimisation
problems. Model Building Optimisation Algorithms (MBOAs), a branch of
evolutionary algorithms, have been successful in combining machine learning
methods and evolutionary search but, until now, they have not utilised DNNs. DO
is the first algorithm to use a DNN to learn and exploit the problem structure
to adapt the variation operator (changing the neighbourhood structure of the
search process). We demonstrate the performance of DO using two theoretical
optimisation problems within the MAXSAT class. The Hierarchical Transformation
Optimisation Problem (HTOP) has controllable deep structure that provides a
clear evaluation of how DO works and why using a layerwise technique is
essential for learning and exploiting problem structure. The Parity Modular
Constraint Problem (MCparity) is a simplistic example of a problem containing
higher-order dependencies (greater than pairwise) which DO can solve and state
of the art MBOAs cannot. Further, we show that DO can exploit deep structure in
TSP instances. Together these results show that there exists problems that DO
can find and exploit deep problem structure that other algorithms cannot.
Making this connection between DNNs and optimisation allows for the utilisation
of advanced tools applicable to DNNs that current MBOAs are unable to use.
