ReNAS:Relativistic Evaluation of Neural Architecture Search
  An effective and efficient architecture performance evaluation scheme is
essential for the success of Neural Architecture Search (NAS). To save
computational cost, most of existing NAS algorithms often train and evaluate
intermediate neural architectures on a small proxy dataset with limited
training epochs. But it is difficult to expect an accurate performance
estimation of an architecture in such a coarse evaluation way. This paper
advocates a new neural architecture evaluation scheme, which aims to determine
which architecture would perform better instead of accurately predict the
absolute architecture performance. Therefore, we propose a
\textbf{relativistic} architecture performance predictor in NAS (ReNAS). We
encode neural architectures into feature tensors, and further refining the
representations with the predictor. The proposed relativistic performance
predictor can be deployed in discrete searching methods to search for the
desired architectures without additional evaluation. Experimental results on
NAS-Bench-101 dataset suggests that, sampling 424 ($0.1\%$ of the entire search
space) neural architectures and their corresponding validation performance is
already enough for learning an accurate architecture performance predictor. The
accuracies of our searched neural architectures on NAS-Bench-101 and
NAS-Bench-201 datasets are higher than that of the state-of-the-art methods and
show the priority of the proposed method.
