It is Not as Good as You Think! Evaluating Simultaneous Machine
  Translation on Interpretation Data
  Most existing simultaneous machine translation (SiMT) systems are trained and
evaluated on offline translation corpora. We argue that SiMT systems should be
trained and tested on real interpretation data. To illustrate this argument, we
propose an interpretation test set and conduct a realistic evaluation of SiMT
trained on offline translations. Our results, on our test set along with 3
existing smaller scale language pairs, highlight the difference of up-to 13.83
BLEU score when SiMT models are evaluated on translation vs interpretation
data. In the absence of interpretation training data, we propose a
translation-to-interpretation (T2I) style transfer method which allows
converting existing offline translations into interpretation-style data,
leading to up-to 2.8 BLEU improvement. However, the evaluation gap remains
notable, calling for constructing large-scale interpretation corpora better
suited for evaluating and developing SiMT systems.
