Dual-State Capsule Networks for Text Classification
  Text classification systems based on contextual embeddings are not viable
options for many of the low resource languages. On the other hand, recently
introduced capsule networks have shown performance in par with these text
classification models. Thus, they could be considered as a viable alternative
for text classification for languages that do not have pre-trained contextual
embedding models. However, current capsule networks depend upon spatial
patterns without considering the sequential features of the text. They are also
sub-optimal in capturing the context-level information in longer sequences.
This paper presents a novel Dual-State Capsule (DS-Caps) network-based
technique for text classification, which is optimized to mitigate these issues.
Two varieties of states, namely sentence-level and word-level, are integrated
with capsule layers to capture deeper context-level information for language
modeling. The dynamic routing process among capsules was also optimized using
the context-level information obtained through sentence-level states. The
DS-Caps networks outperform the existing capsule network architectures for
multiple datasets, particularly for tasks with longer sequences of text. We
also demonstrate the superiority of DS-Caps in text classification for a low
resource language.
