An enhanced Tree-LSTM architecture for sentence semantic modeling using
  typed dependencies
  Tree-based Long short term memory (LSTM) network has become state-of-the-art
for modeling the meaning of language texts as they can effectively exploit the
grammatical syntax and thereby non-linear dependencies among words of the
sentence. However, most of these models cannot recognize the difference in
meaning caused by a change in semantic roles of words or phrases because they
do not acknowledge the type of grammatical relations, also known as typed
dependencies, in sentence structure. This paper proposes an enhanced LSTM
architecture, called relation gated LSTM, which can model the relationship
between two inputs of a sequence using a control input. We also introduce a
Tree-LSTM model called Typed Dependency Tree-LSTM that uses the sentence
dependency parse structure as well as the dependency type to embed sentence
meaning into a dense vector. The proposed model outperformed its type-unaware
counterpart in two typical NLP tasks - Semantic Relatedness Scoring and
Sentiment Analysis, in a lesser number of training epochs. The results were
comparable or competitive with other state-of-the-art models. Qualitative
analysis showed that changes in the voice of sentences had little effect on the
model's predicted scores, while changes in nominal (noun) words had a more
significant impact. The model recognized subtle semantic relationships in
sentence pairs. The magnitudes of learned typed dependencies embeddings were
also in agreement with human intuitions. The research findings imply the
significance of grammatical relations in sentence modeling. The proposed models
would serve as a base for future researches in this direction.
