Pruning Neural Networks with Interpolative Decompositions
  We introduce a principled approach to neural network pruning that casts the
problem as a structured low-rank matrix approximation. Our method uses a novel
application of a matrix factorization technique called the interpolative
decomposition to approximate the activation output of a network layer. This
technique selects neurons or channels in the layer and propagates a corrective
interpolation matrix to the next layer, resulting in a dense, pruned network
with minimal degradation before fine tuning. We demonstrate how to prune a
neural network by first building a set of primitives to prune a single fully
connected or convolution layer and then composing these primitives to prune
deep multi-layer networks. Theoretical guarantees are provided for pruning a
single hidden layer fully connected network. Pruning with interpolative
decompositions achieves strong empirical results compared to the
state-of-the-art on multiple applications from one and two hidden layer
networks on Fashion MNIST to VGG and ResNets on CIFAR-10. Notably, we achieve
an accuracy of 93.62 $\pm$ 0.36% using VGG-16 on CIFAR-10, with a 51% FLOPS
reduction. This gains 0.02% from the full-sized model.
