Benign Overfitting in Adversarially Robust Linear Classification
  "Benign overfitting", where classifiers memorize noisy training data yet
still achieve a good generalization performance, has drawn great attention in
the machine learning community. To explain this surprising phenomenon, a series
of works have provided theoretical justification in over-parameterized linear
regression, classification, and kernel methods. However, it is not clear if
benign overfitting still occurs in the presence of adversarial examples, i.e.,
examples with tiny and intentional perturbations to fool the classifiers. In
this paper, we show that benign overfitting indeed occurs in adversarial
training, a principled approach to defend against adversarial examples. In
detail, we prove the risk bounds of the adversarially trained linear classifier
on the mixture of sub-Gaussian data under $\ell_p$ adversarial perturbations.
Our result suggests that under moderate perturbations, adversarially trained
linear classifiers can achieve the near-optimal standard and adversarial risks,
despite overfitting the noisy training data. Numerical experiments validate our
theoretical findings.
