Human-Paraphrased References Improve Neural Machine Translation
  Automatic evaluation comparing candidate translations to human-generated
paraphrases of reference translations has recently been proposed by Freitag et
al. When used in place of original references, the paraphrased versions produce
metric scores that correlate better with human judgment. This effect holds for
a variety of different automatic metrics, and tends to favor natural
formulations over more literal (translationese) ones. In this paper we compare
the results of performing end-to-end system development using standard and
paraphrased references. With state-of-the-art English-German NMT components, we
show that tuning to paraphrased references produces a system that is
significantly better according to human judgment, but 5 BLEU points worse when
tested on standard references. Our work confirms the finding that paraphrased
references yield metric scores that correlate better with human judgment, and
demonstrates for the first time that using these scores for system development
can lead to significant improvements.
