Distinguishing Non-natural from Natural Adversarial Samples for More
  Robust Pre-trained Language Model
  Recently, the problem of robustness of pre-trained language models (PrLMs)
has received increasing research interest. Latest studies on adversarial
attacks achieve high attack success rates against PrLMs, claiming that PrLMs
are not robust. However, we find that the adversarial samples that PrLMs fail
are mostly non-natural and do not appear in reality. We question the validity
of current evaluation of robustness of PrLMs based on these non-natural
adversarial samples and propose an anomaly detector to evaluate the robustness
of PrLMs with more natural adversarial samples. We also investigate two
applications of the anomaly detector: (1) In data augmentation, we employ the
anomaly detector to force generating augmented data that are distinguished as
non-natural, which brings larger gains to the accuracy of PrLMs. (2) We apply
the anomaly detector to a defense framework to enhance the robustness of PrLMs.
It can be used to defend all types of attacks and achieves higher accuracy on
both adversarial samples and compliant samples than other defense frameworks.
