The Search for Sparse, Robust Neural Networks
  Recent work on deep neural network pruning has shown there exist sparse
subnetworks that achieve equal or improved accuracy, training time, and loss
using fewer network parameters when compared to their dense counterparts.
Orthogonal to pruning literature, deep neural networks are known to be
susceptible to adversarial examples, which may pose risks in security- or
safety-critical applications. Intuition suggests that there is an inherent
trade-off between sparsity and robustness such that these characteristics could
not co-exist. We perform an extensive empirical evaluation and analysis testing
the Lottery Ticket Hypothesis with adversarial training and show this approach
enables us to find sparse, robust neural networks. Code for reproducing
experiments is available here:
https://github.com/justincosentino/robust-sparse-networks.
