Sample-Efficient Neural Architecture Search by Learning Action Space
  Neural Architecture Search (NAS) has emerged as a promising technique for
automatic neural network design. However, existing MCTS based NAS approaches
often utilize manually designed action space, which is not directly related to
the performance metric to be optimized (e.g., accuracy), leading to
sample-inefficient explorations of architectures. To improve the sample
efficiency, this paper proposes Latent Action Neural Architecture Search
(LaNAS), which learns actions to recursively partition the search space into
good or bad regions that contain networks with similar performance metrics.
During the search phase, as different action sequences lead to regions with
different performance, the search efficiency can be significantly improved by
biasing towards the good regions. On three NAS tasks, empirical results
demonstrate that LaNAS is at least an order more sample efficient than baseline
methods including evolutionary algorithms, Bayesian optimizations, and random
search. When applied in practice, both one-shot and regular LaNAS consistently
outperform existing results. Particularly, LaNAS achieves 99.0% accuracy on
CIFAR-10 and 80.8% top1 accuracy at 600 MFLOPS on ImageNet in only 800 samples,
significantly outperforming AmoebaNet with 33x fewer samples. Our code is
publicly available at https://github.com/facebookresearch/LaMCTS.
