Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism
  Principled Robust Deep Neural Nets
  Deep neural nets (DNNs) compression is crucial for adaptation to mobile
devices. Though many successful algorithms exist to compress naturally trained
DNNs, developing efficient and stable compression algorithms for robustly
trained DNNs remains widely open. In this paper, we focus on a co-design of
efficient DNN compression algorithms and sparse neural architectures for robust
and accurate deep learning. Such a co-design enables us to advance the goal of
accommodating both sparsity and robustness. With this objective in mind, we
leverage the relaxed augmented Lagrangian based algorithms to prune the weights
of adversarially trained DNNs, at both structured and unstructured levels.
Using a Feynman-Kac formalism principled robust and sparse DNNs, we can at
least double the channel sparsity of the adversarially trained ResNet20 for
CIFAR10 classification, meanwhile, improve the natural accuracy by $8.69$\% and
the robust accuracy under the benchmark $20$ iterations of IFGSM attack by
$5.42$\%. The code is available at
\url{https://github.com/BaoWangMath/rvsm-rgsm-admm}.
