A Stable Variational Autoencoder for Text Modelling
  Variational Autoencoder (VAE) is a powerful method for learning
representations of high-dimensional data. However, VAEs can suffer from an
issue known as latent variable collapse (or KL loss vanishing), where the
posterior collapses to the prior and the model will ignore the latent codes in
generative tasks. Such an issue is particularly prevalent when employing
VAE-RNN architectures for text modelling (Bowman et al., 2016). In this paper,
we present a simple architecture called holistic regularisation VAE (HR-VAE),
which can effectively avoid latent variable collapse. Compared to existing
VAE-RNN architectures, we show that our model can achieve much more stable
training process and can generate text with significantly better quality.
