Restricted Boltzmann Machines with Gaussian Visible Units Guided by
  Pairwise Constraints
  Restricted Boltzmann machines (RBMs) and their variants are usually trained
by contrastive divergence (CD) learning, but the training procedure is an
unsupervised learning approach, without any guidances of the background
knowledge. To enhance the expression ability of traditional RBMs, in this
paper, we propose pairwise constraints restricted Boltzmann machine with
Gaussian visible units (pcGRBM) model, in which the learning procedure is
guided by pairwise constraints and the process of encoding is conducted under
these guidances. The pairwise constraints are encoded in hidden layer features
of pcGRBM. Then, some pairwise hidden features of pcGRBM flock together and
another part of them are separated by the guidances. In order to deal with
real-valued data, the binary visible units are replaced by linear units with
Gausian noise in the pcGRBM model. In the learning process of pcGRBM, the
pairwise constraints are iterated transitions between visible and hidden units
during CD learning procedure. Then, the proposed model is inferred by
approximative gradient descent method and the corresponding learning algorithm
is designed in this paper. In order to compare the availability of pcGRBM and
traditional RBMs with Gaussian visible units, the features of the pcGRBM and
RBMs hidden layer are used as input 'data' for K-means, spectral clustering
(SP) and affinity propagation (AP) algorithms, respectively. A thorough
experimental evaluation is performed with sixteen image datasets of Microsoft
Research Asia Multimedia (MSRA-MM). The experimental results show that the
clustering performance of K-means, SP and AP algorithms based on pcGRBM model
are significantly better than traditional RBMs. In addition, the pcGRBM model
for clustering task shows better performance than some semi-supervised
clustering algorithms.
