Stopping Criteria in Contrastive Divergence: Alternatives to the
  Reconstruction Error
  Restricted Boltzmann Machines (RBMs) are general unsupervised learning
devices to ascertain generative models of data distributions. RBMs are often
trained using the Contrastive Divergence learning algorithm (CD), an
approximation to the gradient of the data log-likelihood. A simple
reconstruction error is often used to decide whether the approximation provided
by the CD algorithm is good enough, though several authors (Schulz et al.,
2010; Fischer & Igel, 2010) have raised doubts concerning the feasibility of
this procedure. However, not many alternatives to the reconstruction error have
been used in the literature. In this manuscript we investigate simple
alternatives to the reconstruction error in order to detect as soon as possible
the decrease in the log-likelihood during learning.
