Multichannel CNN with Attention for Text Classification
  Recent years, the approaches based on neural networks have shown remarkable
potential for sentence modeling. There are two main neural network structures:
recurrent neural network (RNN) and convolution neural network (CNN). RNN can
capture long term dependencies and store the semantics of the previous
information in a fixed-sized vector. However, RNN is a biased model and its
ability to extract global semantics is restricted by the fixed-sized vector.
Alternatively, CNN is able to capture n-gram features of texts by utilizing
convolutional filters. But the width of convolutional filters restricts its
performance. In order to combine the strengths of the two kinds of networks and
alleviate their shortcomings, this paper proposes Attention-based Multichannel
Convolutional Neural Network (AMCNN) for text classification. AMCNN utilizes a
bi-directional long short-term memory to encode the history and future
information of words into high dimensional representations, so that the
information of both the front and back of the sentence can be fully expressed.
Then the scalar attention and vectorial attention are applied to obtain
multichannel representations. The scalar attention can calculate the word-level
importance and the vectorial attention can calculate the feature-level
importance. In the classification task, AMCNN uses a CNN structure to cpture
word relations on the representations generated by the scalar and vectorial
attention mechanism instead of calculating the weighted sums. It can
effectively extract the n-gram features of the text. The experimental results
on the benchmark datasets demonstrate that AMCNN achieves better performance
than state-of-the-art methods. In addition, the visualization results verify
the semantic richness of multichannel representations.
