Hierarchical Transformer for Multilingual Machine Translation
  The choice of parameter sharing strategy in multilingual machine translation
models determines how optimally parameter space is used and hence, directly
influences ultimate translation quality. Inspired by linguistic trees that show
the degree of relatedness between different languages, the new general approach
to parameter sharing in multilingual machine translation was suggested
recently. The main idea is to use these expert language hierarchies as a basis
for multilingual architecture: the closer two languages are, the more
parameters they share. In this work, we test this idea using the Transformer
architecture and show that despite the success in previous work there are
problems inherent to training such hierarchical models. We demonstrate that in
case of carefully chosen training strategy the hierarchical architecture can
outperform bilingual models and multilingual models with full parameter
sharing.
