What Do Compressed Deep Neural Networks Forget?
  Deep neural network pruning and quantization techniques have demonstrated it
is possible to achieve high levels of compression with surprisingly little
degradation to test set accuracy. However, this measure of performance conceals
significant differences in how different classes and images are impacted by
model compression techniques. We find that models with radically different
numbers of weights have comparable top-line performance metrics but diverge
considerably in behavior on a narrow subset of the dataset. This small subset
of data points, which we term Pruning Identified Exemplars (PIEs) are
systematically more impacted by the introduction of sparsity. Compression
disproportionately impacts model performance on the underrepresented long-tail
of the data distribution. PIEs over-index on atypical or noisy images that are
far more challenging for both humans and algorithms to classify. Our work
provides intuition into the role of capacity in deep neural networks and the
trade-offs incurred by compression. An understanding of this disparate impact
is critical given the widespread deployment of compressed models in the wild.
