The world as a neural network
  We discuss a possibility that the entire universe on its most fundamental
level is a neural network. We identify two different types of dynamical degrees
of freedom: "trainable" variables (e.g. bias vector or weight matrix) and
"hidden" variables (e.g. state vector of neurons). We first consider stochastic
evolution of the trainable variables to argue that near equilibrium their
dynamics is well approximated by Madelung equations (with free energy
representing the phase) and further away from the equilibrium by
Hamilton-Jacobi equations (with free energy representing the Hamilton's
principal function). This shows that the trainable variables can indeed exhibit
classical and quantum behaviors with the state vector of neurons representing
the hidden variables. We then study stochastic evolution of the hidden
variables by considering $D$ non-interacting subsystems with average state
vectors, $\bar{\bf x}^{1}$, ..., $\bar{\bf x}^{D}$ and an overall average state
vector $\bar{\bf x}^{0}$. In the limit when the weight matrix is a permutation
matrix, the dynamics of $\bar{\bf x}^{\mu}$ can be described in terms of
relativistic strings in an emergent $D+1$ dimensional Minkowski space-time. If
the subsystems are minimally interacting, with interactions described by a
metric tensor, then the emergent space-time becomes curved. We argue that the
entropy production in such a system is a local function of the metric tensor
which should be determined by the symmetries of the Onsager tensor. It turns
out that a very simple and highly symmetric Onsager tensor leads to the entropy
production described by the Einstein-Hilbert term. This shows that the learning
dynamics of a neural network can indeed exhibit approximate behaviors described
by both quantum mechanics and general relativity. We also discuss a possibility
that the two descriptions are holographic duals of each other.
