Campfire: Compressible, Regularization-Free, Structured Sparse Training
  for Hardware Accelerators
  This paper studies structured sparse training of CNNs with a gradual pruning
technique that leads to fixed, sparse weight matrices after a set number of
epochs. We simplify the structure of the enforced sparsity so that it reduces
overhead caused by regularization. The proposed training methodology Campfire
explores pruning at granularities within a convolutional kernel and filter.
  We study various tradeoffs with respect to pruning duration, level of
sparsity, and learning rate configuration. We show that our method creates a
sparse version of ResNet-50 and ResNet-50 v1.5 on full ImageNet while remaining
within a negligible <1% margin of accuracy loss. To ensure that this type of
sparse training does not harm the robustness of the network, we also
demonstrate how the network behaves in the presence of adversarial attacks. Our
results show that with 70% target sparsity, over 75% top-1 accuracy is
achievable.
