UENAS: A Unified Evolution-based NAS Framework
  Neural architecture search (NAS) has gained significant attention for
automatic network design in recent years. Previous NAS methods suffer from
limited search spaces, which may lead to sub-optimal results. In this paper, we
propose UENAS, an evolution-based NAS framework with a broader search space
that supports optimizing network architectures, pruning strategies, and
hyperparameters simultaneously. To alleviate the huge search cost caused by the
expanded search space, three strategies are adopted: First, an adaptive pruning
strategy that iteratively trims the average model size in the population
without compromising performance. Second, child networks share weights of
overlapping layers with pre-trained parent networks to reduce the training
epochs. Third, an online predictor scores the joint representations of
architecture, pruning strategy, and hyperparameters to filter out inferior
combos. By the proposed three strategies, the search efficiency is
significantly improved and more well-performed compact networks with tailored
hyper-parameters are derived. In experiments, UENAS achieves error rates of
2.81% on CIFAR-10, 20.24% on CIFAR-100, and 33% on Tiny-ImageNet, which shows
the effectiveness of our method.
