SuperNet in Neural Architecture Search: A Taxonomic Survey
  Deep Neural Networks (DNN) have made significant progress in a wide range of
visual recognition tasks such as image classification, object detection, and
semantic segmentation. The evolution of convolutional architectures has led to
better performance by incurring expensive computational costs. In addition,
network design has become a difficult task, which is labor-intensive and
requires a high level of domain knowledge. To mitigate such issues, there have
been studies for a variety of neural architecture search methods that
automatically search for optimal architectures, achieving models with
impressive performance that outperform human-designed counterparts. This survey
aims to provide an overview of existing works in this field of research and
specifically focus on the supernet optimization that builds a neural network
that assembles all the architectures as its sub models by using weight sharing.
We aim to accomplish that by categorizing supernet optimization by proposing
them as solutions to the common challenges found in the literature: data-side
optimization, poor rank correlation alleviation, and transferable NAS for a
number of deployment scenarios.
