Training robust neural networks using Lipschitz bounds
  Due to their susceptibility to adversarial perturbations, neural networks
(NNs) are hardly used in safety-critical applications. One measure of
robustness to such perturbations in the input is the Lipschitz constant of the
input-output map defined by an NN. In this work, we propose a framework to
train multi-layer NNs while at the same time encouraging robustness by keeping
their Lipschitz constant small, thus addressing the robustness issue. More
specifically, we design an optimization scheme based on the Alternating
Direction Method of Multipliers that minimizes not only the training loss of an
NN but also its Lipschitz constant resulting in a semidefinite programming
based training procedure that promotes robustness. We design two versions of
this training procedure. The first one includes a regularizer that penalizes an
accurate upper bound on the Lipschitz constant. The second one allows to
enforce a desired Lipschitz bound on the NN at all times during training.
Finally, we provide two examples to show that the proposed framework
successfully increases the robustness of NNs.
