Toward Compact Deep Neural Networks via Energy-Aware Pruning
  Despite the remarkable performance, modern deep neural networks are
inevitably accompanied by a significant amount of computational cost for
learning and deployment, which may be incompatible with their usage on edge
devices. Recent efforts to reduce these overheads involve pruning and
decomposing the parameters of various layers without performance deterioration.
Inspired by several decomposition studies, in this paper, we propose a novel
energy-aware pruning method that quantifies the importance of each filter in
the network using nuclear-norm (NN). Proposed energy-aware pruning leads to
state-of-the-art performance for Top-1 accuracy, FLOPs, and parameter reduction
across a wide range of scenarios with multiple network architectures on
CIFAR-10 and ImageNet after fine-grained classification tasks. On toy
experiment, without fine-tuning, we can visually observe that NN has a minute
change in decision boundaries across classes and outperforms the previous
popular criteria. We achieve competitive results with 40.4/49.8% of FLOPs and
45.9/52.9% of parameter reduction with 94.13/94.61% in the Top-1 accuracy with
ResNet-56/110 on CIFAR-10, respectively. In addition, our observations are
consistent for a variety of different pruning setting in terms of data size as
well as data quality which can be emphasized in the stability of the
acceleration and compression with negligible accuracy loss.
