Improving Sequence Modeling Ability of Recurrent Neural Networks via
  Sememes
  Sememes, the minimum semantic units of human languages, have been
successfully utilized in various natural language processing applications.
However, most existing studies exploit sememes in specific tasks and few
efforts are made to utilize sememes more fundamentally. In this paper, we
propose to incorporate sememes into recurrent neural networks (RNNs) to improve
their sequence modeling ability, which is beneficial to all kinds of downstream
tasks. We design three different sememe incorporation methods and employ them
in typical RNNs including LSTM, GRU and their bidirectional variants. In
evaluation, we use several benchmark datasets involving PTB and WikiText-2 for
language modeling, SNLI for natural language inference and another two datasets
for sentiment analysis and paraphrase detection. Experimental results show
evident and consistent improvement of our sememe-incorporated models compared
with vanilla RNNs, which proves the effectiveness of our sememe incorporation
methods. Moreover, we find the sememe-incorporated models have higher
robustness and outperform adversarial training in defending adversarial attack.
All the code and data of this work can be obtained at
https://github.com/thunlp/SememeRNN.
