DARTS-: Robustly Stepping out of Performance Collapse Without Indicators
  Despite the fast development of differentiable architecture search (DARTS),
it suffers from long-standing performance instability, which extremely limits
its application. Existing robustifying methods draw clues from the resulting
deteriorated behavior instead of finding out its causing factor. Various
indicators such as Hessian eigenvalues are proposed as a signal to stop
searching before the performance collapses. However, these indicator-based
methods tend to easily reject good architectures if the thresholds are
inappropriately set, let alone the searching is intrinsically noisy. In this
paper, we undertake a more subtle and direct approach to resolve the collapse.
We first demonstrate that skip connections have a clear advantage over other
candidate operations, where it can easily recover from a disadvantageous state
and become dominant. We conjecture that this privilege is causing degenerated
performance. Therefore, we propose to factor out this benefit with an auxiliary
skip connection, ensuring a fairer competition for all operations. We call this
approach DARTS-. Extensive experiments on various datasets verify that it can
substantially improve robustness. Our code is available at
https://github.com/Meituan-AutoML/DARTS- .
