AutoGrow: Automatic Layer Growing in Deep Convolutional Networks
  Depth is a key component of Deep Neural Networks (DNNs), however, designing
depth is heuristic and requires many human efforts. We propose AutoGrow to
automate depth discovery in DNNs: starting from a shallow seed architecture,
AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops
growing and thus discovers the depth. We propose robust growing and stopping
policies to generalize to different network architectures and datasets. Our
experiments show that by applying the same policy to different network
architectures, AutoGrow can always discover near-optimal depth on various
datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet. For
example, in terms of accuracy-computation trade-off, AutoGrow discovers a
better depth combination in ResNets than human experts. Our AutoGrow is
efficient. It discovers depth within similar time of training a single DNN. Our
code is available at https://github.com/wenwei202/autogrow.
