The Representation Theory of Neural Networks
  In this work, we show that neural networks can be represented via the
mathematical theory of quiver representations. More specifically, we prove that
a neural network is a quiver representation with activation functions, a
mathematical object that we represent using a network quiver. Also, we show
that network quivers gently adapt to common neural network concepts such as
fully-connected layers, convolution operations, residual connections, batch
normalization, pooling operations and even randomly wired neural networks. We
show that this mathematical representation is by no means an approximation of
what neural networks are as it exactly matches reality. This interpretation is
algebraic and can be studied with algebraic methods. We also provide a quiver
representation model to understand how a neural network creates representations
from the data. We show that a neural network saves the data as quiver
representations, and maps it to a geometrical space called the moduli space,
which is given in terms of the underlying oriented graph of the network, i.e.,
its quiver. This results as a consequence of our defined objects and of
understanding how the neural network computes a prediction in a combinatorial
and algebraic way. Overall, representing neural networks through the quiver
representation theory leads to 9 consequences and 4 inquiries for future
research that we believe are of great interest to better understand what neural
networks are and how they work.
