Robust Design of Deep Neural Networks against Adversarial Attacks based
  on Lyapunov Theory
  Deep neural networks (DNNs) are vulnerable to subtle adversarial
perturbations applied to the input. These adversarial perturbations, though
imperceptible, can easily mislead the DNN. In this work, we take a control
theoretic approach to the problem of robustness in DNNs. We treat each
individual layer of the DNN as a nonlinear dynamical system and use Lyapunov
theory to prove stability and robustness locally. We then proceed to prove
stability and robustness globally for the entire DNN. We develop empirically
tight bounds on the response of the output layer, or any hidden layer, to
adversarial perturbations added to the input, or the input of hidden layers.
Recent works have proposed spectral norm regularization as a solution for
improving robustness against l2 adversarial attacks. Our results give new
insights into how spectral norm regularization can mitigate the adversarial
effects. Finally, we evaluate the power of our approach on a variety of data
sets and network architectures and against some of the well-known adversarial
attacks.
