CLIP: Cheap Lipschitz Training of Neural Networks
  Despite the large success of deep neural networks (DNN) in recent years, most
neural networks still lack mathematical guarantees in terms of stability. For
instance, DNNs are vulnerable to small or even imperceptible input
perturbations, so called adversarial examples, that can cause false
predictions. This instability can have severe consequences in applications
which influence the health and safety of humans, e.g., biomedical imaging or
autonomous driving. While bounding the Lipschitz constant of a neural network
improves stability, most methods rely on restricting the Lipschitz constants of
each layer which gives a poor bound for the actual Lipschitz constant.
  In this paper we investigate a variational regularization method named CLIP
for controlling the Lipschitz constant of a neural network, which can easily be
integrated into the training procedure. We mathematically analyze the proposed
model, in particular discussing the impact of the chosen regularization
parameter on the output of the network. Finally, we numerically evaluate our
method on both a nonlinear regression problem and the MNIST and Fashion-MNIST
classification databases, and compare our results with a weight regularization
approach.
