Deep Learning the Ising Model Near Criticality
  It is well established that neural networks with deep architectures perform
better than shallow networks for many tasks in machine learning. In statistical
physics, while there has been recent interest in representing physical data
with generative modelling, the focus has been on shallow neural networks. A
natural question to ask is whether deep neural networks hold any advantage over
shallow networks in representing such data. We investigate this question by
using unsupervised, generative graphical models to learn the probability
distribution of a two-dimensional Ising system. Deep Boltzmann machines, deep
belief networks, and deep restricted Boltzmann networks are trained on thermal
spin configurations from this system, and compared to the shallow architecture
of the restricted Boltzmann machine. We benchmark the models, focussing on the
accuracy of generating energetic observables near the phase transition, where
these quantities are most difficult to approximate. Interestingly, after
training the generative networks, we observe that the accuracy essentially
depends only on the number of neurons in the first hidden layer of the network,
and not on other model details such as network depth or model type. This is
evidence that shallow networks are more efficient than deep networks at
representing physical probability distributions associated with Ising systems
near criticality.
