How to Simplify Search: Classification-wise Pareto Evolution for
  One-shot Neural Architecture Search
  In the deployment of deep neural models, how to effectively and automatically
find feasible deep models under diverse design objectives is fundamental. Most
existing neural architecture search (NAS) methods utilize surrogates to predict
the detailed performance (e.g., accuracy and model size) of a candidate
architecture during the search, which however is complicated and inefficient.
In contrast, we aim to learn an efficient Pareto classifier to simplify the
search process of NAS by transforming the complex multi-objective NAS task into
a simple Pareto-dominance classification task. To this end, we propose a
classification-wise Pareto evolution approach for one-shot NAS, where an online
classifier is trained to predict the dominance relationship between the
candidate and constructed reference architectures, instead of using surrogates
to fit the objective functions. The main contribution of this study is to
change supernet adaption into a Pareto classifier. Besides, we design two
adaptive schemes to select the reference set of architectures for constructing
classification boundary and regulate the rate of positive samples over negative
ones, respectively. We compare the proposed evolution approach with
state-of-the-art approaches on widely-used benchmark datasets, and experimental
results indicate that the proposed approach outperforms other approaches and
have found a number of neural architectures with different model sizes ranging
from 2M to 6M under diverse objectives and constraints.
