Precise Statistical Analysis of Classification Accuracies for
  Adversarial Training
  Despite the wide empirical success of modern machine learning algorithms and
models in a multitude of applications, they are known to be highly susceptible
to seemingly small indiscernible perturbations to the input data known as
\emph{adversarial attacks}. A variety of recent adversarial training procedures
have been proposed to remedy this issue. Despite the success of such procedures
at increasing accuracy on adversarially perturbed inputs or \emph{robust
accuracy}, these techniques often reduce accuracy on natural unperturbed inputs
or \emph{standard accuracy}. Complicating matters further, the effect and trend
of adversarial training procedures on standard and robust accuracy is rather
counter intuitive and radically dependent on a variety of factors including the
perceived form of the perturbation during training, size/quality of data, model
overparameterization, etc. In this paper we focus on binary classification
problems where the data is generated according to the mixture of two Gaussians
with general anisotropic covariance matrices and derive a precise
characterization of the standard and robust accuracy for a class of minimax
adversarially trained models. We consider a general norm-based adversarial
model, where the adversary can add perturbations of bounded $\ell_p$ norm to
each input data, for an arbitrary $p\ge 1$. Our comprehensive analysis allows
us to theoretically explain several intriguing empirical phenomena and provide
a precise understanding of the role of different problem parameters on standard
and robust accuracies.
