The coupling effect of Lipschitz regularization in deep neural networks
  We investigate robustness of deep feed-forward neural networks when input
data are subject to random uncertainties. More specifically, we consider
regularization of the network by its Lipschitz constant and emphasize its role.
We highlight the fact that this regularization is not only a way to control the
magnitude of the weights but has also a coupling effect on the network weights
accross the layers. We claim and show evidence on a dataset that this coupling
effect brings a tradeoff between robustness and expressiveness of the network.
This suggests that Lipschitz regularization should be carefully implemented so
as to maintain coupling accross layers.
