RAPDARTS: Resource-Aware Progressive Differentiable Architecture Search
  Early neural network architectures were designed by so-called "grad student
descent". Since then, the field of Neural Architecture Search (NAS) has
developed with the goal of algorithmically designing architectures tailored for
a dataset of interest. Recently, gradient-based NAS approaches have been
created to rapidly perform the search. Gradient-based approaches impose more
structure on the search, compared to alternative NAS methods, enabling faster
search phase optimization. In the real-world, neural architecture performance
is measured by more than just high accuracy. There is increasing need for
efficient neural architectures, where resources such as model size or latency
must also be considered. Gradient-based NAS is also suitable for such
multi-objective optimization. In this work we extend a popular gradient-based
NAS method to support one or more resource costs. We then perform in-depth
analysis on the discovery of architectures satisfying single-resource
constraints for classification of CIFAR-10.
