Learning to Prune in Training via Dynamic Channel Propagation
  In this paper, we propose a novel network training mechanism called "dynamic
channel propagation" to prune the neural networks during the training period.
In particular, we pick up a specific group of channels in each convolutional
layer to participate in the forward propagation in training time according to
the significance level of channel, which is defined as channel utility. The
utility values with respect to all selected channels are updated simultaneously
with the error back-propagation process and will adaptively change.
Furthermore, when the training ends, channels with high utility values are
retained whereas those with low utility values are discarded. Hence, our
proposed scheme trains and prunes neural networks simultaneously. We
empirically evaluate our novel training scheme on various representative
benchmark datasets and advanced convolutional neural network (CNN)
architectures, including VGGNet and ResNet. The experiment results verify the
superior performance and robust effectiveness of our approach.
