Regularized Sequential Latent Variable Models with Adversarial Neural
  Networks
  The recurrent neural networks (RNN) with richly distributed internal states
and flexible non-linear transition functions, have overtaken the dynamic
Bayesian networks such as the hidden Markov models (HMMs) in the task of
modeling highly structured sequential data. These data, such as from speech and
handwriting, often contain complex relationships between the underlaying
variational factors and the observed data. The standard RNN model has very
limited randomness or variability in its structure, coming from the output
conditional probability model. This paper will present different ways of using
high level latent random variables in RNN to model the variability in the
sequential data, and the training method of such RNN model under the VAE
(Variational Autoencoder) principle. We will explore possible ways of using
adversarial method to train a variational RNN model. Contrary to competing
approaches, our approach has theoretical optimum in the model training and
provides better model training stability. Our approach also improves the
posterior approximation in the variational inference network by a separated
adversarial training step. Numerical results simulated from TIMIT speech data
show that reconstruction loss and evidence lower bound converge to the same
level and adversarial training loss converges to 0.
