Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield
  Networks with Arbitrary Priors
  Restricted Boltzmann Machines are described by the Gibbs measure of a
bipartite spin glass, which in turn corresponds to the one of a generalised
Hopfield network. This equivalence allows us to characterise the state of these
systems in terms of retrieval capabilities, both at low and high load. We study
the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as
the pattern (i.e. weight) distribution and spin (i.e. unit) priors vary
smoothly from Gaussian real variables to Boolean discrete variables. Our
analysis shows that the presence of a retrieval phase is robust and not
peculiar to the standard Hopfield model with Boolean patterns. The retrieval
region is larger when the pattern entries and retrieval units get more peaked
and, conversely, when the hidden units acquire a broader prior and therefore
have a stronger response to high fields. Moreover, at low load retrieval always
exists below some critical temperature, for every pattern distribution ranging
from the Boolean to the Gaussian case.
