Long Short-Term Attention
  Attention is an important cognition process of humans, which helps humans
concentrate on critical information during their perception and learning.
However, although many machine learning models can remember information of
data, they have no the attention mechanism. For example, the long short-term
memory (LSTM) network is able to remember sequential information, but it cannot
pay special attention to part of the sequences. In this paper, we present a
novel model called long short-term attention (LSTA), which seamlessly
integrates the attention mechanism into the inner cell of LSTM. More than
processing long short term dependencies, LSTA can focus on important
information of the sequences with the attention mechanism. Extensive
experiments demonstrate that LSTA outperforms LSTM and related models on the
sequence learning tasks.
