BRP-NAS: Prediction-based NAS using GCNs
  Neural architecture search (NAS) enables researchers to automatically explore
broad design spaces in order to improve efficiency of neural networks. This
efficiency is especially important in the case of on-device deployment, where
improvements in accuracy should be balanced out with computational demands of a
model. In practice, performance metrics of model are computationally expensive
to obtain. Previous work uses a proxy (e.g., number of operations) or a
layer-wise measurement of neural network layers to estimate end-to-end hardware
performance but the imprecise prediction diminishes the quality of NAS. To
address this problem, we propose BRP-NAS, an efficient hardware-aware NAS
enabled by an accurate performance predictor-based on graph convolutional
network (GCN). What is more, we investigate prediction quality on different
metrics and show that sample efficiency of the predictor-based NAS can be
improved by considering binary relations of models and an iterative data
selection strategy. We show that our proposed method outperforms all prior
methods on NAS-Bench-101 and NAS-Bench-201, and that our predictor can
consistently learn to extract useful features from the DARTS search space,
improving upon the second-order baseline. Finally, to raise awareness of the
fact that accurate latency estimation is not a trivial task, we release
LatBench -- a latency dataset of NAS-Bench-201 models running on a broad range
of devices.
