Simple Deep Random Model Ensemble
  Representation learning and unsupervised learning are two central topics of
machine learning and signal processing. Deep learning is one of the most
effective unsupervised representation learning approach. The main contributions
of this paper to the topics are as follows. (i) We propose to view the
representative deep learning approaches as special cases of the knowledge reuse
framework of clustering ensemble. (ii) We propose to view sparse coding when
used as a feature encoder as the consensus function of clustering ensemble, and
view dictionary learning as the training process of the base clusterings of
clustering ensemble. (ii) Based on the above two views, we propose a very
simple deep learning algorithm, named deep random model ensemble (DRME). It is
a stack of random model ensembles. Each random model ensemble is a special
k-means ensemble that discards the expectation-maximization optimization of
each base k-means but only preserves the default initialization method of the
base k-means. (iv) We propose to select the most powerful representation among
the layers by applying DRME to clustering where the single-linkage is used as
the clustering algorithm. Moreover, the DRME based clustering can also detect
the number of the natural clusters accurately. Extensive experimental
comparisons with 5 representation learning methods on 19 benchmark data sets
demonstrate the effectiveness of DRME.
