Disentangled Neural Architecture Search
  Neural architecture search has shown its great potential in various areas
recently. However, existing methods rely heavily on a black-box controller to
search architectures, which suffers from the serious problem of lacking
interpretability. In this paper, we propose disentangled neural architecture
search (DNAS) which disentangles the hidden representation of the controller
into semantically meaningful concepts, making the neural architecture search
process interpretable. Based on systematical study, we discover the correlation
between network architecture and its performance, and propose a dense-sampling
strategy to conduct a targeted search in promising regions that may generate
well-performing architectures. We show that: 1) DNAS successfully disentangles
the architecture representations, including operation selection, skip
connections, and number of layers. 2) Benefiting from interpretability, DNAS
can find excellent architectures under different FLOPS restrictions flexibly.
3) Dense-sampling leads to neural architecture search with higher efficiency
and better performance. On the NASBench-101 dataset, DNAS achieves
state-of-the-art performance of 94.21% using less than 1/13 computational cost
of baseline methods. On ImageNet dataset, DNAS discovers the competitive
architectures that achieves 22.7% test error. our method provides a new
perspective of understanding neural architecture search.
