Reliably fast adversarial training via latent adversarial perturbation
  While multi-step adversarial training is widely popular as an effective
defense method against strong adversarial attacks, its computational cost is
notoriously expensive, compared to standard training. Several single-step
adversarial training methods have been proposed to mitigate the above-mentioned
overhead cost; however, their performance is not sufficiently reliable
depending on the optimization setting. To overcome such limitations, we deviate
from the existing input-space-based adversarial training regime and propose a
single-step latent adversarial training method (SLAT), which leverages the
gradients of latent representation as the latent adversarial perturbation. We
demonstrate that the L1 norm of feature gradients is implicitly regularized
through the adopted latent perturbation, thereby recovering local linearity and
ensuring reliable performance, compared to the existing single-step adversarial
training methods. Because latent perturbation is based on the gradients of the
latent representations which can be obtained for free in the process of input
gradients computation, the proposed method costs roughly the same time as the
fast gradient sign method. Experiment results demonstrate that the proposed
method, despite its structural simplicity, outperforms state-of-the-art
accelerated adversarial training methods.
