{'fraction_non_alphanumeric': 0.020346646571213264, 'fraction_numerical': 0.012057272042200452, 'mean_word_length': 4.981981981981982, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Structured Pruning for Deep Neural Networks with Adaptive Pruning Rate Derivation Based on Connection Sensitivity and Loss Function

— Structured pruning for deep neural networks has been proposed for network model compression. Because earlier structured pruning methods assign pruning rate manually, finding appropriate pruning rate to suppress the degradation of pruned model accuracy is difficult. In this paper, we propose a structured pruning method by deriving pruning rate for each layer adaptively based on gradient and loss function. The proposed method first calculates a threshold of L1-norm of the pruned weight for each layer using loss function and gradient per layer. The threshold guarantees no degradation of the loss function by pruning when the L1-norm of pruned weight is less than that threshold. Then, by comparing the L1-norm of the pruned weight and the calculated threshold while changing the pruning rate, we derive the pruning rate for each layer, which does not degrade the loss function. By applying the derived pruning rate per layer, the accuracy degradation of the pruned model is suppressed. We evaluate the proposed method on CIFAR-10 task with VGG-16 and ResNet in iterative pruning method. For example, our proposed method reduces model parameters of ResNet-56 by 66.3% with 93.71% accuracy.