ESPN: Extremely Sparse Pruned Networks
  Deep neural networks are often highly overparameterized, prohibiting their
use in compute-limited systems. However, a line of recent works has shown that
the size of deep networks can be considerably reduced by identifying a subset
of neuron indicators (or mask) that correspond to significant weights prior to
training. We demonstrate that an simple iterative mask discovery method can
achieve state-of-the-art compression of very deep networks. Our algorithm
represents a hybrid approach between single shot network pruning methods (such
as SNIP) with Lottery-Ticket type approaches. We validate our approach on
several datasets and outperform several existing pruning approaches in both
test accuracy and compression ratio.
