BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of
  Channels
  One-Shot methods have evolved into one of the most popular methods in Neural
Architecture Search (NAS) due to weight sharing and single training of a
supernet. However, existing methods generally suffer from two issues:
predetermined number of channels in each layer which is suboptimal; and model
averaging effects and poor ranking correlation caused by weight coupling and
continuously expanding search space. To explicitly address these issues, in
this paper, a Broadening-and-Shrinking One-Shot NAS (BS-NAS) framework is
proposed, in which `broadening' refers to broadening the search space with a
spring block enabling search for numbers of channels during training of the
supernet; while `shrinking' refers to a novel shrinking strategy gradually
turning off those underperforming operations. The above innovations broaden the
search space for wider representation and then shrink it by gradually removing
underperforming operations, followed by an evolutionary algorithm to
efficiently search for the optimal architecture. Extensive experiments on
ImageNet illustrate the effectiveness of the proposed BS-NAS as well as the
state-of-the-art performance.
