RARTS: a Relaxed Architecture Search Method
  Differentiable architecture search (DARTS) is an effective method for
data-driven neural network design based on solving a bilevel optimization
problem. In this paper, we formulate a single level alternative and a relaxed
architecture search (RARTS) method that utilizes training and validation
datasets in architecture learning without involving mixed second derivatives of
the corresponding loss functions. Through weight/architecture variable
splitting and Gauss-Seidel iterations, the core algorithm outperforms DARTS
significantly in accuracy and search efficiency, as shown in both a solvable
model and CIFAR-10 based architecture search. Our model continues to
out-perform DARTS upon transfer to ImageNet and is on par with recent variants
of DARTS even though our innovation is purely on the training algorithm.
