Fast Gradient Non-sign Methods
  Adversarial attacks make their success in DNNs, and among them,
gradient-based algorithms become one of the mainstreams. Based on the linearity
hypothesis, under $\ell_\infty$ constraint, $sign$ operation applied to the
gradients is a good choice for generating perturbations. However, side-effects
from such operation exist since it leads to the bias of direction between real
gradients and perturbations. In other words, current methods contain a gap
between real gradients and actual noises, which leads to biased and inefficient
attacks. Therefore in this paper, based on the Taylor expansion, the bias is
analyzed theoretically, and the correction of $sign$, i.e., Fast Gradient
Non-sign Method (FGNM), is further proposed. Notably, FGNM is a general routine
that seamlessly replaces the conventional $sign$ operation in gradient-based
attacks with negligible extra computational cost. Extensive experiments
demonstrate the effectiveness of our methods. Specifically, for untargeted
black-box attacks, ours outperform them by 27.5% at most and 9.5% on average.
For targeted attacks against defense models, it is 15.1% and 12.7%. Our
anonymous code is publicly available at https://github.com/yaya-cheng/FGNM
