Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial
  Robustness
  Evaluating the robustness of a defense model is a challenging task in
adversarial robustness research. Obfuscated gradients, a type of gradient
masking, have previously been found to exist in many defense methods and cause
a false signal of robustness. In this paper, we identify a more subtle
situation called Imbalanced Gradients that can also cause overestimated
adversarial robustness. The phenomenon of imbalanced gradients occurs when the
gradient of one term of the margin loss dominates and pushes the attack towards
to a suboptimal direction. To exploit imbalanced gradients, we formulate a
Margin Decomposition (MD) attack that decomposes a margin loss into individual
terms and then explores the attackability of these terms separately via a
two-stage process. We also propose a MultiTargeted and an ensemble version of
our MD attack. By investigating 17 defense models proposed since 2018, we find
that 6 models are susceptible to imbalanced gradients and our MD attack can
decrease their robustness evaluated by the best baseline standalone attack by
another 2%. We also provide an in-depth analysis of the likely causes of
imbalanced gradients and effective countermeasures.
