Variational Information Bottleneck for Unsupervised Clustering: Deep
  Gaussian Mixture Embedding
  In this paper, we develop an unsupervised generative clustering framework
that combines the Variational Information Bottleneck and the Gaussian Mixture
Model. Specifically, in our approach, we use the Variational Information
Bottleneck method and model the latent space as a mixture of Gaussians. We
derive a bound on the cost function of our model that generalizes the Evidence
Lower Bound (ELBO) and provide a variational inference type algorithm that
allows computing it. In the algorithm, the coders' mappings are parametrized
using neural networks, and the bound is approximated by Monte Carlo sampling
and optimized with stochastic gradient descent. Numerical results on real
datasets are provided to support the efficiency of our method.
