FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding
  Inducing latent tree structures from sequential data is an emerging trend in
the NLP research landscape today, largely popularized by recent methods such as
Gumbel LSTM and Ordered Neurons (ON-LSTM). This paper proposes FASTTREES, a new
general purpose neural module for fast sequence encoding. Unlike most previous
works that consider recurrence to be necessary for tree induction, our work
explores the notion of parallel tree induction, i.e., imbuing our model with
hierarchical inductive biases in a parallelizable, non-autoregressive fashion.
To this end, our proposed FASTTREES achieves competitive or superior
performance to ON-LSTM on four well-established sequence modeling tasks, i.e.,
language modeling, logical inference, sentiment analysis and natural language
inference. Moreover, we show that the FASTTREES module can be applied to
enhance Transformer models, achieving performance gains on three sequence
transduction tasks (machine translation, subject-verb agreement and
mathematical language understanding), paving the way for modular tree induction
modules. Overall, we outperform existing state-of-the-art models on logical
inference tasks by +4% and mathematical language understanding by +8%.
