SCOP: Scientific Control for Reliable Neural Network Pruning
  This paper proposes a reliable neural network pruning algorithm by setting up
a scientific control. Existing pruning methods have developed various
hypotheses to approximate the importance of filters to the network and then
execute filter pruning accordingly. To increase the reliability of the results,
we prefer to have a more rigorous research design by including a scientific
control group as an essential part to minimize the effect of all factors except
the association between the filter and expected network output. Acting as a
control group, knockoff feature is generated to mimic the feature map produced
by the network filter, but they are conditionally independent of the example
label given the real feature map. We theoretically suggest that the knockoff
condition can be approximately preserved given the information propagation of
network layers. Besides the real feature map on an intermediate layer, the
corresponding knockoff feature is brought in as another auxiliary input signal
for the subsequent layers. Redundant filters can be discovered in the
adversarial process of different features. Through experiments, we demonstrate
the superiority of the proposed algorithm over state-of-the-art methods. For
example, our method can reduce 57.8% parameters and 60.2% FLOPs of ResNet-101
with only 0.01% top-1 accuracy loss on ImageNet. The code is available at
https://github.com/huawei-noah/Pruning/tree/master/SCOP_NeurIPS2020.
