{'fraction_non_alphanumeric': 0.022263450834879406, 'fraction_numerical': 0.0, 'mean_word_length': 5.7698744769874475, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Improved Trade-off Between Accuracy and Complexity with Progressive Gradient Pruning

Although deep neural networks (NNs) have achieved state-of-the-art accuracy in many visual recognition tasks, the growing computational complexity and energy consumption of networks remains an issue, especially for applications on platforms with limited resources and requiring real-time processing. Filter pruning techniques have recently shown promising results for the compression of convolutional NNs (CNNs). However, these techniques involve numerous steps and complex optimisations because some only prune after training CNNs, while others prune from scratch during training by integrating sparsity constraints or modifying the loss function. The progressive soft filter pruning (PSFP) technique provides greater training efficiency, but its soft pruning strategy does not handle the backward pass, i.e. momentum pruning, which is needed for better optimization. We proposed a new Progressive Gradient Pruning (PGP) technique for iterative filter pruning during training. To improve on PSFP, it relies on a novel filter selection criterion that measures the change in filter weights, and new hard and soft pruning strategies to effectively adapt momentum tensors during the backward propagation pass. Experimental results obtained after training various CNNs on image data for classification and object detection benchmarks indicate that the PGP technique can achieve a better tradeoff between classification accuracy and network (time and memory) complexity than PSFP and other state-of-the-art filter pruning techniques.