Label Smoothing and Logit Squeezing: A Replacement for Adversarial
  Training?
  Adversarial training is one of the strongest defenses against adversarial
attacks, but it requires adversarial examples to be generated for every
mini-batch during optimization. The expense of producing these examples during
training often precludes adversarial training from use on complex image
datasets. In this study, we explore the mechanisms by which adversarial
training improves classifier robustness, and show that these mechanisms can be
effectively mimicked using simple regularization methods, including label
smoothing and logit squeezing. Remarkably, using these simple regularization
methods in combination with Gaussian noise injection, we are able to achieve
strong adversarial robustness -- often exceeding that of adversarial training
-- using no adversarial examples.
