Bridging the Performance Gap between FGSM and PGD Adversarial Training
  Deep learning achieves state-of-the-art performance in many tasks but exposes
to the underlying vulnerability against adversarial examples. Across existing
defense techniques, adversarial training with the projected gradient decent
attack (adv.PGD) is considered as one of the most effective ways to achieve
moderate adversarial robustness. However, adv.PGD requires too much training
time since the projected gradient attack (PGD) takes multiple iterations to
generate perturbations. On the other hand, adversarial training with the fast
gradient sign method (adv.FGSM) takes much less training time since the fast
gradient sign method (FGSM) takes one step to generate perturbations but fails
to increase adversarial robustness. In this work, we extend adv.FGSM to make it
achieve the adversarial robustness of adv.PGD. We demonstrate that the large
curvature along FGSM perturbed direction leads to a large difference in
performance of adversarial robustness between adv.FGSM and adv.PGD, and
therefore propose combining adv.FGSM with a curvature regularization
(adv.FGSMR) in order to bridge the performance gap between adv.FGSM and
adv.PGD. The experiments show that adv.FGSMR has higher training efficiency
than adv.PGD. In addition, it achieves comparable performance of adversarial
robustness on MNIST dataset under white-box attack, and it achieves better
performance than adv.PGD under white-box attack and effectively defends the
transferable adversarial attack on CIFAR-10 dataset.
