LOss-Based SensiTivity rEgulaRization: towards deep sparse neural
  networks
  LOBSTER (LOss-Based SensiTivity rEgulaRization) is a method for training
neural networks having a sparse topology. Let the sensitivity of a network
parameter be the variation of the loss function with respect to the variation
of the parameter. Parameters with low sensitivity, i.e. having little impact on
the loss when perturbed, are shrunk and then pruned to sparsify the network.
Our method allows to train a network from scratch, i.e. without preliminary
learning or rewinding. Experiments on multiple architectures and datasets show
competitive compression ratios with minimal computational overhead.
