Accurate, reliable and fast robustness evaluation
  Throughout the past five years, the susceptibility of neural networks to
minimal adversarial perturbations has moved from a peculiar phenomenon to a
core issue in Deep Learning. Despite much attention, however, progress towards
more robust models is significantly impaired by the difficulty of evaluating
the robustness of neural network models. Today's methods are either fast but
brittle (gradient-based attacks), or they are fairly reliable but slow (score-
and decision-based attacks). We here develop a new set of gradient-based
adversarial attacks which (a) are more reliable in the face of gradient-masking
than other gradient-based attacks, (b) perform better and are more query
efficient than current state-of-the-art gradient-based attacks, (c) can be
flexibly adapted to a wide range of adversarial criteria and (d) require
virtually no hyperparameter tuning. These findings are carefully validated
across a diverse set of six different models and hold for L0, L1, L2 and Linf
in both targeted as well as untargeted scenarios. Implementations will soon be
available in all major toolboxes (Foolbox, CleverHans and ART). We hope that
this class of attacks will make robustness evaluations easier and more
reliable, thus contributing to more signal in the search for more robust
machine learning models.
