On Network Design Spaces for Visual Recognition
  Over the past several years progress in designing better neural network
architectures for visual recognition has been substantial. To help sustain this
rate of progress, in this work we propose to reexamine the methodology for
comparing network architectures. In particular, we introduce a new comparison
paradigm of distribution estimates, in which network design spaces are compared
by applying statistical techniques to populations of sampled models, while
controlling for confounding factors like network complexity. Compared to
current methodologies of comparing point and curve estimates of model families,
distribution estimates paint a more complete picture of the entire design
landscape. As a case study, we examine design spaces used in neural
architecture search (NAS). We find significant statistical differences between
recent NAS design space variants that have been largely overlooked.
Furthermore, our analysis reveals that the design spaces for standard model
families like ResNeXt can be comparable to the more complex ones used in recent
NAS work. We hope these insights into distribution analysis will enable more
robust progress toward discovering better networks for visual recognition.
