(Pen-) Ultimate DNN Pruning
  DNN pruning reduces memory footprint and computational work of DNN-based
solutions to improve performance and energy-efficiency. An effective pruning
scheme should be able to systematically remove connections and/or neurons that
are unnecessary or redundant, reducing the DNN size without any loss in
accuracy. In this paper we show that prior pruning schemes require an extremely
time-consuming iterative process that requires retraining the DNN many times to
tune the pruning hyperparameters. We propose a DNN pruning scheme based on
Principal Component Analysis and relative importance of each neuron's
connection that automatically finds the optimized DNN in one shot without
requiring hand-tuning of multiple parameters.
