Multi-view Sentence Representation Learning
  Multi-view learning can provide self-supervision when different views are
available of the same data. The distributional hypothesis provides another form
of useful self-supervision from adjacent sentences which are plentiful in large
unlabelled corpora. Motivated by the asymmetry in the two hemispheres of the
human brain as well as the observation that different learning architectures
tend to emphasise different aspects of sentence meaning, we create a unified
multi-view sentence representation learning framework, in which, one view
encodes the input sentence with a Recurrent Neural Network (RNN), and the other
view encodes it with a simple linear model, and the training objective is to
maximise the agreement specified by the adjacent context information between
two views. We show that, after training, the vectors produced from our
multi-view training provide improved representations over the single-view
training, and the combination of different views gives further representational
improvement and demonstrates solid transferability on standard downstream
tasks.
