CONet: Channel Optimization for Convolutional Neural Networks
  Neural Architecture Search (NAS) has shifted network design from using human
intuition to leveraging search algorithms guided by evaluation metrics. We
study channel size optimization in convolutional neural networks (CNN) and
identify the role it plays in model accuracy and complexity. Current channel
size selection methods are generally limited by discrete sample spaces while
suffering from manual iteration and simple heuristics. To solve this, we
introduce an efficient dynamic scaling algorithm -- CONet -- that automatically
optimizes channel sizes across network layers for a given CNN. Two metrics --
"\textit{Rank}" and "\textit{Rank Average Slope}" -- are introduced to identify
the information accumulated in training. The algorithm dynamically scales
channel sizes up or down over a fixed searching phase. We conduct experiments
on CIFAR10/100 and ImageNet datasets and show that CONet can find efficient and
accurate architectures searched in ResNet, DARTS, and DARTS+ spaces that
outperform their baseline models.
  This document supersedes previously published paper in ICCV2021-NeurArch
workshop. An additional section is included on manual scaling of channel size
in CNNs to numerically validate of the metrics used in searching optimum
channel configurations in CNNs.
