Semantic Label Smoothing for Sequence to Sequence Problems
  Label smoothing has been shown to be an effective regularization strategy in
classification, that prevents overfitting and helps in label de-noising.
However, extending such methods directly to seq2seq settings, such as Machine
Translation, is challenging: the large target output space of such problems
makes it intractable to apply label smoothing over all possible outputs. Most
existing approaches for seq2seq settings either do token level smoothing, or
smooth over sequences generated by randomly substituting tokens in the target
sequence. Unlike these works, in this paper, we propose a technique that
smooths over \emph{well formed} relevant sequences that not only have
sufficient n-gram overlap with the target sequence, but are also
\emph{semantically similar}. Our method shows a consistent and significant
improvement over the state-of-the-art techniques on different datasets.
