Accelerating Multi-Objective Neural Architecture Search by Random-Weight
  Evaluation
  For the goal of automated design of high-performance deep convolutional
neural networks (CNNs), Neural Architecture Search (NAS) methodology is
becoming increasingly important for both academia and industries.Due to the
costly stochastic gradient descent (SGD) training of CNNs for performance
evaluation, most existing NAS methods are computationally expensive for
real-world deployments. To address this issue, we first introduce a new
performance estimation metric, named Random-Weight Evaluation (RWE) to quantify
the quality of CNNs in a cost-efficient manner. Instead of fully training the
entire CNN, the RWE only trains its last layer and leaves the remainders with
randomly initialized weights, which results in a single network evaluation in
seconds.Second, a complexity metric is adopted for multi-objective NAS to
balance the model size and performance. Overall, our proposed method obtains a
set of efficient models with state-of-the-art performance in two real-world
search spaces. Then the results obtained on the CIFAR-10 dataset are
transferred to the ImageNet dataset to validate the practicality of the
proposed algorithm. Moreover, ablation studies on NAS-Bench-301 datasets reveal
the effectiveness of the proposed RWE in estimating the performance compared
with existing methods.
