Nonnegative Restricted Boltzmann Machines for Parts-based
  Representations Discovery and Predictive Model Stabilization
  The success of any machine learning system depends critically on effective
representations of data. In many cases, it is desirable that a representation
scheme uncovers the parts-based, additive nature of the data. Of current
representation learning schemes, restricted Boltzmann machines (RBMs) have
proved to be highly effective in unsupervised settings. However, when it comes
to parts-based discovery, RBMs do not usually produce satisfactory results. We
enhance such capacity of RBMs by introducing nonnegativity into the model
weights, resulting in a variant called nonnegative restricted Boltzmann machine
(NRBM). The NRBM produces not only controllable decomposition of data into
interpretable parts but also offers a way to estimate the intrinsic nonlinear
dimensionality of data, and helps to stabilize linear predictive models. We
demonstrate the capacity of our model on applications such as handwritten digit
recognition, face recognition, document classification and patient readmission
prognosis. The decomposition quality on images is comparable with or better
than what produced by the nonnegative matrix factorization (NMF), and the
thematic features uncovered from text are qualitatively interpretable in a
similar manner to that of the latent Dirichlet allocation (LDA). The stability
performance of feature selection on medical data is better than RBM and
competitive with NMF. The learned features, when used for classification, are
more discriminative than those discovered by both NMF and LDA and comparable
with those by RBM.
