A Probabilistic Formulation of Unsupervised Text Style Transfer
  We present a deep generative model for unsupervised text style transfer that
unifies previously proposed non-generative techniques. Our probabilistic
approach models non-parallel data from two domains as a partially observed
parallel corpus. By hypothesizing a parallel latent sequence that generates
each observed sequence, our model learns to transform sequences from one domain
to another in a completely unsupervised fashion. In contrast with traditional
generative sequence models (e.g. the HMM), our model makes few assumptions
about the data it generates: it uses a recurrent language model as a prior and
an encoder-decoder as a transduction distribution. While computation of
marginal data likelihood is intractable in this model class, we show that
amortized variational inference admits a practical surrogate. Further, by
drawing connections between our variational objective and other recent
unsupervised style transfer and machine translation techniques, we show how our
probabilistic view can unify some known non-generative objectives such as
backtranslation and adversarial loss. Finally, we demonstrate the effectiveness
of our method on a wide range of unsupervised style transfer tasks, including
sentiment transfer, formality transfer, word decipherment, author imitation,
and related language translation. Across all style transfer tasks, our approach
yields substantial gains over state-of-the-art non-generative baselines,
including the state-of-the-art unsupervised machine translation techniques that
our approach generalizes. Further, we conduct experiments on a standard
unsupervised machine translation task and find that our unified approach
matches the current state-of-the-art.
