MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep
  Networks
  We present MorphNet, an approach to automate the design of neural network
structures. MorphNet iteratively shrinks and expands a network, shrinking via a
resource-weighted sparsifying regularizer on activations and expanding via a
uniform multiplicative factor on all layers. In contrast to previous
approaches, our method is scalable to large networks, adaptable to specific
resource constraints (e.g. the number of floating-point operations per
inference), and capable of increasing the network's performance. When applied
to standard network architectures on a wide variety of datasets, our approach
discovers novel structures in each domain, obtaining higher performance while
respecting the resource constraint.
