Adversarially Robust Generalization Just Requires More Unlabeled Data
  Neural network robustness has recently been highlighted by the existence of
adversarial examples. Many previous works show that the learned networks do not
perform well on perturbed test data, and significantly more labeled data is
required to achieve adversarially robust generalization. In this paper, we
theoretically and empirically show that with just more unlabeled data, we can
learn a model with better adversarially robust generalization. The key insight
of our results is based on a risk decomposition theorem, in which the expected
robust risk is separated into two parts: the stability part which measures the
prediction stability in the presence of perturbations, and the accuracy part
which evaluates the standard classification accuracy. As the stability part
does not depend on any label information, we can optimize this part using
unlabeled data. We further prove that for a specific Gaussian mixture problem,
adversarially robust generalization can be almost as easy as the standard
generalization in supervised learning if a sufficiently large amount of
unlabeled data is provided. Inspired by the theoretical findings, we further
show that a practical adversarial training algorithm that leverages unlabeled
data can improve adversarial robust generalization on MNIST and Cifar-10.
