RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting
  and Output Merging
  Pruning Deep Neural Networks (DNNs) is a prominent field of study in the goal
of inference runtime acceleration. In this paper, we introduce a novel
data-free pruning protocol RED++. Only requiring a trained neural network, and
not specific to DNN architecture, we exploit an adaptive data-free scalar
hashing which exhibits redundancies among neuron weight values. We study the
theoretical and empirical guarantees on the preservation of the accuracy from
the hashing as well as the expected pruning ratio resulting from the
exploitation of said redundancies. We propose a novel data-free pruning
technique of DNN layers which removes the input-wise redundant operations. This
algorithm is straightforward, parallelizable and offers novel perspective on
DNN pruning by shifting the burden of large computation to efficient memory
access and allocation. We provide theoretical guarantees on RED++ performance
and empirically demonstrate its superiority over other data-free pruning
methods and its competitiveness with data-driven ones on ResNets, MobileNets
and EfficientNets.
