Restricted Boltzmann Machine and Deep Belief Network: Tutorial and
  Survey
  This is a tutorial and survey paper on Boltzmann Machine (BM), Restricted
Boltzmann Machine (RBM), and Deep Belief Network (DBN). We start with the
required background on probabilistic graphical models, Markov random field,
Gibbs sampling, statistical physics, Ising model, and the Hopfield network.
Then, we introduce the structures of BM and RBM. The conditional distributions
of visible and hidden variables, Gibbs sampling in RBM for generating
variables, training BM and RBM by maximum likelihood estimation, and
contrastive divergence are explained. Then, we discuss different possible
discrete and continuous distributions for the variables. We introduce
conditional RBM and how it is trained. Finally, we explain deep belief network
as a stack of RBM models. This paper on Boltzmann machines can be useful in
various fields including data science, statistics, neural computation, and
statistical physics.
