Improving Differentiable Architecture Search with a Generative Model
  In differentiable neural architecture search (NAS) algorithms like DARTS, the
training set used to update model weight and the validation set used to update
model architectures are sampled from the same data distribution. Thus, the
uncommon features in the dataset fail to receive enough attention during
training. In this paper, instead of introducing more complex NAS algorithms, we
explore the idea that adding quality synthesized datasets into training can
help the classification model identify its weakness and improve recognition
accuracy. We introduce a training strategy called ``Differentiable Architecture
Search with a Generative Model(DASGM)." In DASGM, the training set is used to
update the classification model weight, while a synthesized dataset is used to
train its architecture. The generated images have different distributions from
the training set, which can help the classification model learn better features
to identify its weakness. We formulate DASGM into a multi-level optimization
framework and develop an effective algorithm to solve it. Experiments on
CIFAR-10, CIFAR-100, and ImageNet have demonstrated the effectiveness of DASGM.
Code will be made available.
