Improving Text Generation with Student-Forcing Optimal Transport
  Neural language models are often trained with maximum likelihood estimation
(MLE), where the next word is generated conditioned on the ground-truth word
tokens. During testing, however, the model is instead conditioned on previously
generated tokens, resulting in what is termed exposure bias. To reduce this gap
between training and testing, we propose using optimal transport (OT) to match
the sequences generated in these two modes. An extension is further proposed to
improve the OT learning, based on the structural and contextual information of
the text sequences. The effectiveness of the proposed method is validated on
machine translation, text summarization, and text generation tasks.
