Adversarial Risk Bounds for Neural Networks through Sparsity based
  Compression
  Neural networks have been shown to be vulnerable against minor adversarial
perturbations of their inputs, especially for high dimensional data under
$\ell_\infty$ attacks. To combat this problem, techniques like adversarial
training have been employed to obtain models which are robust on the training
set. However, the robustness of such models against adversarial perturbations
may not generalize to unseen data. To study how robustness generalizes, recent
works assume that the inputs have bounded $\ell_2$-norm in order to bound the
adversarial risk for $\ell_\infty$ attacks with no explicit dimension
dependence. In this work we focus on $\ell_\infty$ attacks on $\ell_\infty$
bounded inputs and prove margin-based bounds. Specifically, we use a
compression based approach that relies on efficiently compressing the set of
tunable parameters without distorting the adversarial risk. To achieve this, we
apply the concept of effective sparsity and effective joint sparsity on the
weight matrices of neural networks. This leads to bounds with no explicit
dependence on the input dimension, neither on the number of classes. Our
results show that neural networks with approximately sparse weight matrices not
only enjoy enhanced robustness, but also better generalization.
