L$^{2}$NAS: Learning to Optimize Neural Architectures via
  Continuous-Action Reinforcement Learning
  Neural architecture search (NAS) has achieved remarkable results in deep
neural network design. Differentiable architecture search converts the search
over discrete architectures into a hyperparameter optimization problem which
can be solved by gradient descent. However, questions have been raised
regarding the effectiveness and generalizability of gradient methods for
solving non-convex architecture hyperparameter optimization problems. In this
paper, we propose L$^{2}$NAS, which learns to intelligently optimize and update
architecture hyperparameters via an actor neural network based on the
distribution of high-performing architectures in the search history. We
introduce a quantile-driven training procedure which efficiently trains
L$^{2}$NAS in an actor-critic framework via continuous-action reinforcement
learning. Experiments show that L$^{2}$NAS achieves state-of-the-art results on
NAS-Bench-201 benchmark as well as DARTS search space and Once-for-All
MobileNetV3 search space. We also show that search policies generated by
L$^{2}$NAS are generalizable and transferable across different training
datasets with minimal fine-tuning.
