Discretization-Aware Architecture Search
  The search cost of neural architecture search (NAS) has been largely reduced
by weight-sharing methods. These methods optimize a super-network with all
possible edges and operations, and determine the optimal sub-network by
discretization, \textit{i.e.}, pruning off weak candidates. The discretization
process, performed on either operations or edges, incurs significant inaccuracy
and thus the quality of the final architecture is not guaranteed. This paper
presents discretization-aware architecture search (DA\textsuperscript{2}S),
with the core idea being adding a loss term to push the super-network towards
the configuration of desired topology, so that the accuracy loss brought by
discretization is largely alleviated. Experiments on standard image
classification benchmarks demonstrate the superiority of our approach, in
particular, under imbalanced target network configurations that were not
studied before.
