Refining Source Representations with Relation Networks for Neural
  Machine Translation
  Although neural machine translation (NMT) with the encoder-decoder framework
has achieved great success in recent times, it still suffers from some
drawbacks: RNNs tend to forget old information which is often useful and the
encoder only operates through words without considering word relationship. To
solve these problems, we introduce a relation networks (RN) into NMT to refine
the encoding representations of the source. In our method, the RN first
augments the representation of each source word with its neighbors and reasons
all the possible pairwise relations between them. Then the source
representations and all the relations are fed to the attention module and the
decoder together, keeping the main encoder-decoder architecture unchanged.
Experiments on two Chinese-to-English data sets in different scales both show
that our method can outperform the competitive baselines significantly.
