Joint Optimization of an Autoencoder for Clustering and Embedding
  Deep embedded clustering has become a dominating approach to unsupervised
categorization of objects with deep neural networks. The optimization of the
most popular methods alternates between the training of a deep autoencoder and
a k-means clustering of the autoencoder's embedding. The diachronic setting,
however, prevents the former to benefit from valuable information acquired by
the latter. In this paper, we present an alternative where the autoencoder and
the clustering are learned simultaneously. This is achieved by providing novel
theoretical insight, where we show that the objective function of a certain
class of Gaussian mixture models (GMMs) can naturally be rephrased as the loss
function of a one-hidden layer autoencoder thus inheriting the built-in
clustering capabilities of the GMM. That simple neural network, referred to as
the clustering module, can be integrated into a deep autoencoder resulting in a
deep clustering model able to jointly learn a clustering and an embedding.
Experiments confirm the equivalence between the clustering module and Gaussian
mixture models. Further evaluations affirm the empirical relevance of our deep
architecture as it outperforms related baselines on several data sets.
