Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial
  Training
  Delusive attacks aim to substantially deteriorate the test accuracy of the
learning model by slightly perturbing the features of correctly labeled
training examples. By formalizing this malicious attack as finding the
worst-case training data within a specific $\infty$-Wasserstein ball, we show
that minimizing adversarial risk on the perturbed data is equivalent to
optimizing an upper bound of natural risk on the original data. This implies
that adversarial training can serve as a principled defense against delusive
attacks. Thus, the test accuracy decreased by delusive attacks can be largely
recovered by adversarial training. To further understand the internal mechanism
of the defense, we disclose that adversarial training can resist the delusive
perturbations by preventing the learner from overly relying on non-robust
features in a natural setting. Finally, we complement our theoretical findings
with a set of experiments on popular benchmark datasets, which show that the
defense withstands six different practical attacks. Both theoretical and
empirical results vote for adversarial training when confronted with delusive
adversaries.
