Group Pruning using a Bounded-Lp norm for Group Gating and
  Regularization
  Deep neural networks achieve state-of-the-art results on several tasks while
increasing in complexity. It has been shown that neural networks can be pruned
during training by imposing sparsity inducing regularizers. In this paper, we
investigate two techniques for group-wise pruning during training in order to
improve network efficiency. We propose a gating factor after every
convolutional layer to induce channel level sparsity, encouraging insignificant
channels to become exactly zero. Further, we introduce and analyse a bounded
variant of the L1 regularizer, which interpolates between L1 and L0-norms to
retain performance of the network at higher pruning rates. To underline
effectiveness of the proposed methods,we show that the number of parameters of
ResNet-164, DenseNet-40 and MobileNetV2 can be reduced down by 30%, 69% and 75%
on CIFAR100 respectively without a significant drop in accuracy. We achieve
state-of-the-art pruning results for ResNet-50 with higher accuracy on
ImageNet. Furthermore, we show that the light weight MobileNetV2 can further be
compressed on ImageNet without a significant drop in performance.
