A novel channel pruning method for deep neural network compression
  In recent years, deep neural networks have achieved great success in the
field of computer vision. However, it is still a big challenge to deploy these
deep models on resource-constrained embedded devices such as mobile robots,
smart phones and so on. Therefore, network compression for such platforms is a
reasonable solution to reduce memory consumption and computation complexity. In
this paper, a novel channel pruning method based on genetic algorithm is
proposed to compress very deep Convolution Neural Networks (CNNs). Firstly, a
pre-trained CNN model is pruned layer by layer according to the sensitivity of
each layer. After that, the pruned model is fine-tuned based on knowledge
distillation framework. These two improvements significantly decrease the model
redundancy with less accuracy drop. Channel selection is a combinatorial
optimization problem that has exponential solution space. In order to
accelerate the selection process, the proposed method formulates it as a search
problem, which can be solved efficiently by genetic algorithm. Meanwhile, a
two-step approximation fitness function is designed to further improve the
efficiency of genetic process. The proposed method has been verified on three
benchmark datasets with two popular CNN models: VGGNet and ResNet. On the
CIFAR-100 and ImageNet datasets, our approach outperforms several
state-of-the-art methods. On the CIFAR-10 and SVHN datasets, the pruned VGGNet
achieves better performance than the original model with 8 times parameters
compression and 3 times FLOPs reduction.
