Dynamical Variational Autoencoders: A Comprehensive Review
  The Variational Autoencoder (VAE) is a powerful deep generative model that is
now extensively used to represent high-dimensional complex data via a
low-dimensional latent space learned in an unsupervised manner. In the original
VAE model, input data vectors are processed independently. In recent years, a
series of papers have presented different extensions of the VAE to process
sequential data, that not only model the latent space, but also model the
temporal dependencies within a sequence of data vectors and corresponding
latent vectors, relying on recurrent neural networks or state space models. In
this paper we perform an extensive literature review of these models.
Importantly, we introduce and discuss a general class of models called
Dynamical Variational Autoencoders (DVAEs) that encompasses a large subset of
these temporal VAE extensions. Then we present in detail seven different
instances of DVAE that were recently proposed in the literature, with an effort
to homogenize the notations and presentation lines, as well as to relate these
models with existing classical temporal models. We reimplemented those seven
DVAE models and we present the results of an experimental benchmark conducted
on the speech analysis-resynthesis task (the PyTorch code is made publicly
available). The paper is concluded with an extensive discussion on important
issues concerning the DVAE class of models and future research guidelines.
