On the Effectiveness of Interval Bound Propagation for Training
  Verifiably Robust Models
  Recent work has shown that it is possible to train deep neural networks that
are provably robust to norm-bounded adversarial perturbations. Most of these
methods are based on minimizing an upper bound on the worst-case loss over all
possible adversarial perturbations. While these techniques show promise, they
often result in difficult optimization procedures that remain hard to scale to
larger networks. Through a comprehensive analysis, we show how a simple
bounding technique, interval bound propagation (IBP), can be exploited to train
large provably robust neural networks that beat the state-of-the-art in
verified accuracy. While the upper bound computed by IBP can be quite weak for
general networks, we demonstrate that an appropriate loss and clever
hyper-parameter schedule allow the network to adapt such that the IBP bound is
tight. This results in a fast and stable learning algorithm that outperforms
more sophisticated methods and achieves state-of-the-art results on MNIST,
CIFAR-10 and SVHN. It also allows us to train the largest model to be verified
beyond vacuous bounds on a downscaled version of ImageNet.
