Regularizers for Single-step Adversarial Training
  The progress in the last decade has enabled machine learning models to
achieve impressive performance across a wide range of tasks in Computer Vision.
However, a plethora of works have demonstrated the susceptibility of these
models to adversarial samples. Adversarial training procedure has been proposed
to defend against such adversarial attacks. Adversarial training methods
augment mini-batches with adversarial samples, and typically single-step
(non-iterative) methods are used for generating these adversarial samples.
However, models trained using single-step adversarial training converge to
degenerative minima where the model merely appears to be robust. The pseudo
robustness of these models is due to the gradient masking effect. Although
multi-step adversarial training helps to learn robust models, they are hard to
scale due to the use of iterative methods for generating adversarial samples.
To address these issues, we propose three different types of regularizers that
help to learn robust models using single-step adversarial training methods. The
proposed regularizers mitigate the effect of gradient masking by harnessing on
properties that differentiate a robust model from that of a pseudo robust
model. Performance of models trained using the proposed regularizers is on par
with models trained using computationally expensive multi-step adversarial
training methods.
