MixPath: A Unified Approach for One-shot Neural Architecture Search
  Blending multiple convolutional kernels is proved advantageous in neural
architectural design. However, current neural architecture search approaches
are mainly limited to stacked single-path search space. How can the one-shot
doctrine search for multi-path models remains unresolved. Specifically, we are
motivated to train a multi-path supernet to accurately evaluate the candidate
architectures. In this paper, we discover that in the studied search space,
feature vectors summed from multiple paths are nearly multiples of those from a
single path, which perturbs supernet training and its ranking ability. In this
regard, we propose a novel mechanism called Shadow Batch Normalization(SBN) to
regularize the disparate feature statistics. Extensive experiments prove that
SBN is capable of stabilizing the training and improving the ranking
performance (e.g. Kendall Tau 0.597 tested on NAS-Bench-101). We call our
unified multi-path one-shot approach as MixPath, which generates a series of
models that achieve state-of-the-art results on ImageNet.
