Is Deep Learning a Renormalization Group Flow?
  Although there has been a rapid development of practical applications,
theoretical explanations of deep learning are in their infancy. Deep learning
performs a sophisticated coarse graining. Since coarse graining is a key
ingredient of the renormalization group (RG), RG may provide a useful
theoretical framework directly relevant to deep learning. In this study we
pursue this possibility. A statistical mechanics model for a magnet, the Ising
model, is used to train an unsupervised restricted Boltzmann machine (RBM). The
patterns generated by the trained RBM are compared to the configurations
generated through an RG treatment of the Ising model. Although we are motivated
by the connection between deep learning and RG flow, in this study we focus
mainly on comparing a single layer of a deep network to a single step in the RG
flow. We argue that correlation functions between hidden and visible neurons
are capable of diagnosing RG-like coarse graining. Numerical experiments show
the presence of RG-like patterns in correlators computed using the trained
RBMs. The observables we consider are also able to exhibit important
differences between RG and deep learning.
