Homogeneous Architecture Augmentation for Neural Predictor
  Neural Architecture Search (NAS) can automatically design well-performed
architectures of Deep Neural Networks (DNNs) for the tasks at hand. However,
one bottleneck of NAS is the prohibitively computational cost largely due to
the expensive performance evaluation. The neural predictors can directly
estimate the performance without any training of the DNNs to be evaluated, thus
have drawn increasing attention from researchers. Despite their popularity,
they also suffer a severe limitation: the shortage of annotated DNN
architectures for effectively training the neural predictors. In this paper, we
proposed Homogeneous Architecture Augmentation for Neural Predictor (HAAP) of
DNN architectures to address the issue aforementioned. Specifically, a
homogeneous architecture augmentation algorithm is proposed in HAAP to generate
sufficient training data taking the use of homogeneous representation.
Furthermore, the one-hot encoding strategy is introduced into HAAP to make the
representation of DNN architectures more effective. The experiments have been
conducted on both NAS-Benchmark-101 and NAS-Bench-201 dataset. The experimental
results demonstrate that the proposed HAAP algorithm outperforms the state of
the arts compared, yet with much less training data. In addition, the ablation
studies on both benchmark datasets have also shown the universality of the
homogeneous architecture augmentation.
