Multi-objective Neural Architecture Search with Almost No Training
  In the recent past, neural architecture search (NAS) has attracted increasing
attention from both academia and industries. Despite the steady stream of
impressive empirical results, most existing NAS algorithms are computationally
prohibitive to execute due to the costly iterations of stochastic gradient
descent (SGD) training. In this work, we propose an effective alternative,
dubbed Random-Weight Evaluation (RWE), to rapidly estimate the performance of
network architectures. By just training the last linear classification layer,
RWE reduces the computational cost of evaluating an architecture from hours to
seconds. When integrated within an evolutionary multi-objective algorithm, RWE
obtains a set of efficient architectures with state-of-the-art performance on
CIFAR-10 with less than two hours' searching on a single GPU card. Ablation
studies on rank-order correlations and transfer learning experiments to
ImageNet have further validated the effectiveness of RWE.
