A Reinforced Generation of Adversarial Examples for Neural Machine
  Translation
  Neural machine translation systems tend to fail on less decent inputs despite
its significant efficacy, which may significantly harm the credibility of this
systems-fathoming how and when neural-based systems fail in such cases is
critical for industrial maintenance. Instead of collecting and analyzing bad
cases using limited handcrafted error features, here we investigate this issue
by generating adversarial examples via a new paradigm based on reinforcement
learning. Our paradigm could expose pitfalls for a given performance metric,
e.g., BLEU, and could target any given neural machine translation architecture.
We conduct experiments of adversarial attacks on two mainstream neural machine
translation architectures, RNN-search, and Transformer. The results show that
our method efficiently produces stable attacks with meaning-preserving
adversarial examples. We also present a qualitative and quantitative analysis
for the preference pattern of the attack, demonstrating its capability of
pitfall exposure.
