Multi-Objective Neural Architecture Search Based on Diverse Structures
  and Adaptive Recommendation
  The search space of neural architecture search (NAS) for convolutional neural
network (CNN) is huge. To reduce searching cost, most NAS algorithms use fixed
outer network level structure, and search the repeatable cell structure only.
Such kind of fixed architecture performs well when enough cells and channels
are used. However, when the architecture becomes more lightweight, the
performance decreases significantly. To obtain better lightweight
architectures, more flexible and diversified neural architectures are in
demand, and more efficient methods should be designed for larger search space.
Motivated by this, we propose MoARR algorithm, which utilizes the existing
research results and historical information to quickly find architectures that
are both lightweight and accurate. We use the discovered high-performance cells
to construct network architectures. This method increases the network
architecture diversity while also reduces the search space of cell structure
design. In addition, we designs a novel multi-objective method to effectively
analyze the historical evaluation information, so as to efficiently search for
the Pareto optimal architectures with high accuracy and small parameter number.
Experimental results show that our MoARR can achieve a powerful and lightweight
model (with 1.9% error rate and 2.3M parameters) on CIFAR-10 in 6 GPU hours,
which is better than the state-of-the-arts. The explored architecture is
transferable to ImageNet and achieves 76.0% top-1 accuracy with 4.9M
parameters.
