Training Flexible Depth Model by Multi-Task Learning for Neural Machine
  Translation
  The standard neural machine translation model can only decode with the same
depth configuration as training. Restricted by this feature, we have to deploy
models of various sizes to maintain the same translation latency, because the
hardware conditions on different terminal devices (e.g., mobile phones) may
vary greatly. Such individual training leads to increased model maintenance
costs and slower model iterations, especially for the industry. In this work,
we propose to use multi-task learning to train a flexible depth model that can
adapt to different depth configurations during inference. Experimental results
show that our approach can simultaneously support decoding in 24 depth
configurations and is superior to the individual training and another flexible
depth model training method -- LayerDrop.
