Efficient Evaluation of the Partition Function of RBMs with Annealed
  Importance Sampling
  Probabilistic models based on Restricted Boltzmann Machines (RBMs) imply the
evaluation of normalized Boltzmann factors, which in turn require from the
evaluation of the partition function Z. The exact evaluation of Z, though,
becomes a forbiddingly expensive task as the system size increases. This even
worsens when one considers most usual learning algorithms for RBMs, where the
exact evaluation of the gradient of the log-likelihood of the empirical
distribution of the data includes the computation of Z at each iteration. The
Annealed Importance Sampling (AIS) method provides a tool to stochastically
estimate the partition function of the system. So far, the standard use of the
AIS algorithm in the Machine Learning context has been done using a large
number of Monte Carlo steps. In this work we show that this may not be required
if a proper starting probability distribution is employed as the initialization
of the AIS algorithm. We analyze the performance of AIS in both small- and
large-sized problems, and show that in both cases a good estimation of Z can be
obtained with little computational cost.
