Bilingual is At Least Monolingual (BALM): A Novel Translation Algorithm
  that Encodes Monolingual Priors
  State-of-the-art machine translation (MT) models do not use knowledge of any
single language's structure; this is the equivalent of asking someone to
translate from English to German while knowing neither language. BALM is a
framework incorporates monolingual priors into an MT pipeline; by casting input
and output languages into embedded space using BERT, we can solve machine
translation with much simpler models. We find that English-to-German
translation on the Multi30k dataset can be solved with a simple feedforward
network under the BALM framework with near-SOTA BLEU scores.
