Memory-Augmented Neural Networks for Machine Translation
  Memory-augmented neural networks (MANNs) have been shown to outperform other
recurrent neural network architectures on a series of artificial sequence
learning tasks, yet they have had limited application to real-world tasks. We
evaluate direct application of Neural Turing Machines (NTM) and Differentiable
Neural Computers (DNC) to machine translation. We further propose and evaluate
two models which extend the attentional encoder-decoder with capabilities
inspired by memory augmented neural networks. We evaluate our proposed models
on IWSLT Vietnamese to English and ACL Romanian to English datasets. Our
proposed models and the memory augmented neural networks perform similarly to
the attentional encoder-decoder on the Vietnamese to English translation task
while have a 0.3-1.9 lower BLEU score for the Romanian to English task.
Interestingly, our analysis shows that despite being equipped with additional
flexibility and being randomly initialized memory augmented neural networks
learn an algorithm for machine translation almost identical to the attentional
encoder-decoder.
