Leveraging Filter Correlations for Deep Model Compression
  We present a filter correlation based model compression approach for deep
convolutional neural networks. Our approach iteratively identifies pairs of
filters with the largest pairwise correlations and drops one of the filters
from each such pair. However, instead of discarding one of the filters from
each such pair na\"{i}vely, the model is re-optimized to make the filters in
these pairs maximally correlated, so that discarding one of the filters from
the pair results in minimal information loss. Moreover, after discarding the
filters in each round, we further finetune the model to recover from the
potential small loss incurred by the compression. We evaluate our proposed
approach using a comprehensive set of experiments and ablation studies. Our
compression method yields state-of-the-art FLOPs compression rates on various
benchmarks, such as LeNet-5, VGG-16, and ResNet-50,56, while still achieving
excellent predictive performance for tasks such as object detection on
benchmark datasets.
