Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial
  Robustness
  The vulnerability of deep neural networks to adversarial examples has
motivated an increasing number of defense strategies for promoting model
robustness. However, the progress is usually hampered by insufficient
robustness evaluations. As the de facto standard to evaluate adversarial
robustness, adversarial attacks typically solve an optimization problem of
crafting adversarial examples with an iterative process. In this work, we
propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger
attack algorithms automatically. Our method learns the optimizer in adversarial
attacks parameterized by a recurrent neural network, which is trained over a
class of data samples and defenses to produce effective update directions
during adversarial example generation. Furthermore, we develop a model-agnostic
training algorithm to improve the generalization ability of the learned
optimizer when attacking unseen defenses. Our approach can be flexibly
incorporated with various attacks and consistently improves the performance
with little extra computational cost. Extensive experiments demonstrate the
effectiveness of the learned attacks by MAMA compared to the state-of-the-art
attacks on different defenses, leading to a more reliable evaluation of
adversarial robustness.
