Lookahead: A Far-Sighted Alternative of Magnitude-based Pruning
  Magnitude-based pruning is one of the simplest methods for pruning neural
networks. Despite its simplicity, magnitude-based pruning and its variants
demonstrated remarkable performances for pruning modern architectures. Based on
the observation that magnitude-based pruning indeed minimizes the Frobenius
distortion of a linear operator corresponding to a single layer, we develop a
simple pruning method, coined lookahead pruning, by extending the single layer
optimization to a multi-layer optimization. Our experimental results
demonstrate that the proposed method consistently outperforms magnitude-based
pruning on various networks, including VGG and ResNet, particularly in the
high-sparsity regime. See https://github.com/alinlab/lookahead_pruning for
codes.
