Exploring Complicated Search Spaces with Interleaving-Free Sampling
  The existing neural architecture search algorithms are mostly working on
search spaces with short-distance connections. We argue that such designs,
though safe and stable, obstacles the search algorithms from exploring more
complicated scenarios. In this paper, we build the search algorithm upon a
complicated search space with long-distance connections, and show that existing
weight-sharing search algorithms mostly fail due to the existence of
\textbf{interleaved connections}. Based on the observation, we present a simple
yet effective algorithm named \textbf{IF-NAS}, where we perform a periodic
sampling strategy to construct different sub-networks during the search
procedure, avoiding the interleaved connections to emerge in any of them. In
the proposed search space, IF-NAS outperform both random sampling and previous
weight-sharing search algorithms by a significant margin. IF-NAS also
generalizes to the micro cell-based spaces which are much easier. Our research
emphasizes the importance of macro structure and we look forward to further
efforts along this direction.
