Neural Networks and Quantum Field Theory
  We propose a theoretical understanding of neural networks in terms of
Wilsonian effective field theory. The correspondence relies on the fact that
many asymptotic neural networks are drawn from Gaussian processes, the analog
of non-interacting field theories. Moving away from the asymptotic limit yields
a non-Gaussian process and corresponds to turning on particle interactions,
allowing for the computation of correlation functions of neural network outputs
with Feynman diagrams. Minimal non-Gaussian process likelihoods are determined
by the most relevant non-Gaussian terms, according to the flow in their
coefficients induced by the Wilsonian renormalization group. This yields a
direct connection between overparameterization and simplicity of neural network
likelihoods. Whether the coefficients are constants or functions may be
understood in terms of GP limit symmetries, as expected from 't Hooft's
technical naturalness. General theoretical calculations are matched to neural
network experiments in the simplest class of models allowing the
correspondence. Our formalism is valid for any of the many architectures that
becomes a GP in an asymptotic limit, a property preserved under certain types
of training.
