Domain, Translationese and Noise in Synthetic Data for Neural Machine
  Translation
  The quality of neural machine translation can be improved by leveraging
additional monolingual resources to create synthetic training data. Source-side
monolingual data can be (forward-)translated into the target language for
self-training; target-side monolingual data can be back-translated. It has been
widely reported that back-translation delivers superior results, but could this
be due to artefacts in the test sets? We perform a case study using
French-English news translation task and separate test sets based on their
original languages. We show that forward translation delivers superior gains in
terms of BLEU on sentences that were originally in the source language,
complementing previous studies which show large improvements with
back-translation on sentences that were originally in the target language. To
better understand when and why forward and back-translation are effective, we
study the role of domains, translationese, and noise. While translationese
effects are well known to influence MT evaluation, we also find evidence that
news data from different languages shows subtle domain differences, which is
another explanation for varying performance on different portions of the test
set. We perform additional low-resource experiments which demonstrate that
forward translation is more sensitive to the quality of the initial translation
system than back-translation, and tends to perform worse in low-resource
settings.
