A Neighbourhood-Based Stopping Criterion for Contrastive Divergence
  Learning
  Restricted Boltzmann Machines (RBMs) are general unsupervised learning
devices to ascertain generative models of data distributions. RBMs are often
trained using the Contrastive Divergence learning algorithm (CD), an
approximation to the gradient of the data log-likelihood. A simple
reconstruction error is often used as a stopping criterion for CD, although
several authors
\cite{schulz-et-al-Convergence-Contrastive-Divergence-2010-NIPSw,
fischer-igel-Divergence-Contrastive-Divergence-2010-ICANN} have raised doubts
concerning the feasibility of this procedure. In many cases the evolution curve
of the reconstruction error is monotonic while the log-likelihood is not, thus
indicating that the former is not a good estimator of the optimal stopping
point for learning. However, not many alternatives to the reconstruction error
have been discussed in the literature. In this manuscript we investigate simple
alternatives to the reconstruction error, based on the inclusion of information
contained in neighboring states to the training set, as a stopping criterion
for CD learning.
