WPNAS: Neural Architecture Search by jointly using Weight Sharing and
  Predictor
  Weight sharing based and predictor based methods are two major types of fast
neural architecture search methods. In this paper, we propose to jointly use
weight sharing and predictor in a unified framework. First, we construct a
SuperNet in a weight-sharing way and probabilisticly sample architectures from
the SuperNet. To increase the correctness of the evaluation of architectures,
besides direct evaluation using the inherited weights, we further apply a
few-shot predictor to assess the architecture on the other hand. The final
evaluation of the architecture is the combination of direct evaluation, the
prediction from the predictor and the cost of the architecture. We regard the
evaluation as a reward and apply a self-critical policy gradient approach to
update the architecture probabilities. To further reduce the side effects of
weight sharing, we propose a weakly weight sharing method by introducing
another HyperNet. We conduct experiments on datasets including CIFAR-10,
CIFAR-100 and ImageNet under NATS-Bench, DARTS and MobileNet search space. The
proposed WPNAS method achieves state-of-the-art performance on these datasets.
