ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting
  We propose ResRep, a novel method for lossless channel pruning (a.k.a. filter
pruning), which slims down a CNN by reducing the width (number of output
channels) of convolutional layers. Inspired by the neurobiology research about
the independence of remembering and forgetting, we propose to re-parameterize a
CNN into the remembering parts and forgetting parts, where the former learn to
maintain the performance and the latter learn to prune. Via training with
regular SGD on the former but a novel update rule with penalty gradients on the
latter, we realize structured sparsity. Then we equivalently merge the
remembering and forgetting parts into the original architecture with narrower
layers. In this sense, ResRep can be viewed as a successful application of
Structural Re-parameterization. Such a methodology distinguishes ResRep from
the traditional learning-based pruning paradigm that applies a penalty on
parameters to produce sparsity, which may suppress the parameters essential for
the remembering. ResRep slims down a standard ResNet-50 with 76.15% accuracy on
ImageNet to a narrower one with only 45% FLOPs and no accuracy drop, which is
the first to achieve lossless pruning with such a high compression ratio. The
code and models are at https://github.com/DingXiaoH/ResRep.
