Mean-field inference methods for neural networks
  Machine learning algorithms relying on deep neural networks recently allowed
a great leap forward in artificial intelligence. Despite the popularity of
their applications, the efficiency of these algorithms remains largely
unexplained from a theoretical point of view. The mathematical description of
learning problems involves very large collections of interacting random
variables, difficult to handle analytically as well as numerically. This
complexity is precisely the object of study of statistical physics. Its
mission, originally pointed towards natural systems, is to understand how
macroscopic behaviors arise from microscopic laws. Mean-field methods are one
type of approximation strategy developed in this view. We review a selection of
classical mean-field methods and recent progress relevant for inference in
neural networks. In particular, we remind the principles of derivations of
high-temperature expansions, the replica method and message passing algorithms,
highlighting their equivalences and complementarities. We also provide
references for past and current directions of research on neural networks
relying on mean-field methods.
