Learning Latent Representations in Neural Networks for Clustering
  through Pseudo Supervision and Graph-based Activity Regularization
  In this paper, we propose a novel unsupervised clustering approach exploiting
the hidden information that is indirectly introduced through a pseudo
classification objective. Specifically, we randomly assign a pseudo
parent-class label to each observation which is then modified by applying the
domain specific transformation associated with the assigned label. Generated
pseudo observation-label pairs are subsequently used to train a neural network
with Auto-clustering Output Layer (ACOL) that introduces multiple softmax nodes
for each pseudo parent-class. Due to the unsupervised objective based on
Graph-based Activity Regularization (GAR) terms, softmax duplicates of each
parent-class are specialized as the hidden information captured through the
help of domain specific transformations is propagated during training.
Ultimately we obtain a k-means friendly latent representation. Furthermore, we
demonstrate how the chosen transformation type impacts performance and helps
propagate the latent information that is useful in revealing unknown clusters.
Our results show state-of-the-art performance for unsupervised clustering tasks
on MNIST, SVHN and USPS datasets, with the highest accuracies reported to date
in the literature.
