Geometry-aware Instance-reweighted Adversarial Training
  In adversarial machine learning, there was a common belief that robustness
and accuracy hurt each other. The belief was challenged by recent studies where
we can maintain the robustness and improve the accuracy. However, the other
direction, whether we can keep the accuracy while improving the robustness, is
conceptually and practically more interesting, since robust accuracy should be
lower than standard accuracy for any model. In this paper, we show this
direction is also promising. Firstly, we find even over-parameterized deep
networks may still have insufficient model capacity, because adversarial
training has an overwhelming smoothing effect. Secondly, given limited model
capacity, we argue adversarial data should have unequal importance:
geometrically speaking, a natural data point closer to/farther from the class
boundary is less/more robust, and the corresponding adversarial data point
should be assigned with larger/smaller weight. Finally, to implement the idea,
we propose geometry-aware instance-reweighted adversarial training, where the
weights are based on how difficult it is to attack a natural data point.
Experiments show that our proposal boosts the robustness of standard
adversarial training; combining two directions, we improve both robustness and
accuracy of standard adversarial training.
