Systematic Weight Pruning of DNNs using Alternating Direction Method of
  Multipliers
  We present a systematic weight pruning framework of deep neural networks
(DNNs) using the alternating direction method of multipliers (ADMM). We first
formulate the weight pruning problem of DNNs as a constrained nonconvex
optimization problem, and then adopt the ADMM framework for systematic weight
pruning. We show that ADMM is highly suitable for weight pruning due to the
computational efficiency it offers. We achieve a much higher compression ratio
compared with prior work while maintaining the same test accuracy, together
with a faster convergence rate. Our models are released at
https://github.com/KaiqiZhang/admm-pruning
