Modeling Hierarchical Structures with Continuous Recursive Neural
  Networks
  Recursive Neural Networks (RvNNs), which compose sequences according to their
underlying hierarchical syntactic structure, have performed well in several
natural language processing tasks compared to similar models without structural
biases. However, traditional RvNNs are incapable of inducing the latent
structure in a plain text sequence on their own. Several extensions have been
proposed to overcome this limitation. Nevertheless, these extensions tend to
rely on surrogate gradients or reinforcement learning at the cost of higher
bias or variance. In this work, we propose Continuous Recursive Neural Network
(CRvNN) as a backpropagation-friendly alternative to address the aforementioned
limitations. This is done by incorporating a continuous relaxation to the
induced structure. We demonstrate that CRvNN achieves strong performance in
challenging synthetic tasks such as logical inference and ListOps. We also show
that CRvNN performs comparably or better than prior latent structure models on
real-world tasks such as sentiment analysis and natural language inference.
