Barriers and Dynamical Paths in Alternating Gibbs Sampling of Restricted
  Boltzmann Machines
  Restricted Boltzmann Machines (RBM) are bi-layer neural networks used for the
unsupervised learning of model distributions from data. The bipartite
architecture of RBM naturally defines an elegant sampling procedure, called
Alternating Gibbs Sampling (AGS), where the configurations of the
latent-variable layer are sampled conditional to the data-variable layer, and
vice versa. We study here the performance of AGS on several analytically
tractable models borrowed from statistical mechanics. We show that standard AGS
is not more efficient than classical Metropolis-Hastings (MH) sampling of the
effective energy landscape defined on the data layer. However, RBM can identify
meaningful representations of training data in their latent space. Furthermore,
using these representations and combining Gibbs sampling with the MH algorithm
in the latent space can enhance the sampling performance of the RBM when the
hidden units encode weakly dependent features of the data. We illustrate our
findings on three datasets: Bars and Stripes and MNIST, well known in machine
learning, and the so-called Lattice Proteins, introduced in theoretical biology
to study the sequence-to-structure mapping in proteins.
