Accelerating Neural Architecture Search via Proxy Data
  Despite the increasing interest in neural architecture search (NAS), the
significant computational cost of NAS is a hindrance to researchers. Hence, we
propose to reduce the cost of NAS using proxy data, i.e., a representative
subset of the target data, without sacrificing search performance. Even though
data selection has been used across various fields, our evaluation of existing
selection methods for NAS algorithms offered by NAS-Bench-1shot1 reveals that
they are not always appropriate for NAS and a new selection method is
necessary. By analyzing proxy data constructed using various selection methods
through data entropy, we propose a novel proxy data selection method tailored
for NAS. To empirically demonstrate the effectiveness, we conduct thorough
experiments across diverse datasets, search spaces, and NAS algorithms.
Consequently, NAS algorithms with the proposed selection discover architectures
that are competitive with those obtained using the entire dataset. It
significantly reduces the search cost: executing DARTS with the proposed
selection requires only 40 minutes on CIFAR-10 and 7.5 hours on ImageNet with a
single GPU. Additionally, when the architecture searched on ImageNet using the
proposed selection is inversely transferred to CIFAR-10, a state-of-the-art
test error of 2.4\% is yielded. Our code is available at
https://github.com/nabk89/NAS-with-Proxy-data.
