Neural Architecture Generator Optimization
  Neural Architecture Search (NAS) was first proposed to achieve
state-of-the-art performance through the discovery of new architecture
patterns, without human intervention. An over-reliance on expert knowledge in
the search space design has however led to increased performance (local optima)
without significant architectural breakthroughs, thus preventing truly novel
solutions from being reached. In this work we 1) are the first to investigate
casting NAS as a problem of finding the optimal network generator and 2) we
propose a new, hierarchical and graph-based search space capable of
representing an extremely large variety of network types, yet only requiring
few continuous hyper-parameters. This greatly reduces the dimensionality of the
problem, enabling the effective use of Bayesian Optimisation as a search
strategy. At the same time, we expand the range of valid architectures,
motivating a multi-objective learning approach. We demonstrate the
effectiveness of this strategy on six benchmark datasets and show that our
search space generates extremely lightweight yet highly competitive models.
