An Improving Framework of regularization for Network Compression
  Deep Neural Networks have achieved remarkable success relying on the
developing high computation capability of GPUs and large-scale datasets with
increasing network depth and width in image recognition, object detection and
many other applications. However, due to the expensive computation and
intensive memory, researchers have concentrated on designing compression
methods in recent years. In this paper, we briefly summarize the existing
advanced techniques that are useful in model compression at first. After that,
we give a detailed description on group lasso regularization and its variants.
More importantly, we propose an improving framework of partial regularization
based on the relationship between neurons and connections of adjacent layers.
It is reasonable and feasible with the help of permutation property of neural
network . Experiment results show that partial regularization methods brings
improvements such as higher classification accuracy in both training and
testing stages on multiple datasets. Since our regularizers contain the
computation of less parameters, it shows competitive performances in terms of
the total running time of experiments. Finally, we analysed the results and
draw a conclusion that the optimal network structure must exist and depend on
the input data.
