Variational Mixture of Normalizing Flows
  In the past few years, deep generative models, such as generative adversarial
networks \autocite{GAN}, variational autoencoders \autocite{vaepaper}, and
their variants, have seen wide adoption for the task of modelling complex data
distributions. In spite of the outstanding sample quality achieved by those
early methods, they model the target distributions \emph{implicitly}, in the
sense that the probability density functions induced by them are not explicitly
accessible. This fact renders those methods unfit for tasks that require, for
example, scoring new instances of data with the learned distributions.
Normalizing flows have overcome this limitation by leveraging the
change-of-variables formula for probability density functions, and by using
transformations designed to have tractable and cheaply computable Jacobians.
Although flexible, this framework lacked (until recently
\autocites{semisuplearning_nflows, RAD}) a way to introduce discrete structure
(such as the one found in mixtures) in the models it allows to construct, in an
unsupervised scenario. The present work overcomes this by using normalizing
flows as components in a mixture model and devising an end-to-end training
procedure for such a model. This procedure is based on variational inference,
and uses a variational posterior parameterized by a neural network. As will
become clear, this model naturally lends itself to (multimodal) density
estimation, semi-supervised learning, and clustering. The proposed model is
illustrated on two synthetic datasets, as well as on a real-world dataset.
Keywords: Deep generative models, normalizing flows, variational inference,
probabilistic modelling, mixture models.
