Dirichlet Pruning for Neural Network Compression
  We introduce Dirichlet pruning, a novel post-processing technique to
transform a large neural network model into a compressed one. Dirichlet pruning
is a form of structured pruning that assigns the Dirichlet distribution over
each layer's channels in convolutional layers (or neurons in fully-connected
layers) and estimates the parameters of the distribution over these units using
variational inference. The learned distribution allows us to remove unimportant
units, resulting in a compact architecture containing only crucial features for
a task at hand. The number of newly introduced Dirichlet parameters is only
linear in the number of channels, which allows for rapid training, requiring as
little as one epoch to converge. We perform extensive experiments, in
particular on larger architectures such as VGG and ResNet (45% and 58%
compression rate, respectively) where our method achieves the state-of-the-art
compression performance and provides interpretable features as a by-product.
