Pruning Convolutional Filters using Batch Bridgeout
  State-of-the-art computer vision models are rapidly increasing in capacity,
where the number of parameters far exceeds the number required to fit the
training set. This results in better optimization and generalization
performance. However, the huge size of contemporary models results in large
inference costs and limits their use on resource-limited devices. In order to
reduce inference costs, convolutional filters in trained neural networks could
be pruned to reduce the run-time memory and computational requirements during
inference. However, severe post-training pruning results in degraded
performance if the training algorithm results in dense weight vectors. We
propose the use of Batch Bridgeout, a sparsity inducing stochastic
regularization scheme, to train neural networks so that they could be pruned
efficiently with minimal degradation in performance. We evaluate the proposed
method on common computer vision models VGGNet, ResNet, and Wide-ResNet on the
CIFAR image classification task. For all the networks, experimental results
show that Batch Bridgeout trained networks achieve higher accuracy across a
wide range of pruning intensities compared to Dropout and weight decay
regularization.
