On the mapping between Hopfield networks and Restricted Boltzmann
  Machines
  Hopfield networks (HNs) and Restricted Boltzmann Machines (RBMs) are two
important models at the interface of statistical physics, machine learning, and
neuroscience. Recently, there has been interest in the relationship between HNs
and RBMs, due to their similarity under the statistical mechanics formalism. An
exact mapping between HNs and RBMs has been previously noted for the special
case of orthogonal (uncorrelated) encoded patterns. We present here an exact
mapping in the case of correlated pattern HNs, which are more broadly
applicable to existing datasets. Specifically, we show that any HN with $N$
binary variables and $p<N$ arbitrary binary patterns can be transformed into an
RBM with $N$ binary visible variables and $p$ gaussian hidden variables. We
outline the conditions under which the reverse mapping exists, and conduct
experiments on the MNIST dataset which suggest the mapping provides a useful
initialization to the RBM weights. We discuss extensions, the potential
importance of this correspondence for the training of RBMs, and for
understanding the performance of deep architectures which utilize RBMs.
