Deep Neural Networks Are Congestion Games: From Loss Landscape to
  Wardrop Equilibrium and Beyond
  The theoretical analysis of deep neural networks (DNN) is arguably among the
most challenging research directions in machine learning (ML) right now, as it
requires from scientists to lay novel statistical learning foundations to
explain their behaviour in practice. While some success has been achieved
recently in this endeavour, the question on whether DNNs can be analyzed using
the tools from other scientific fields outside the ML community has not
received the attention it may well have deserved. In this paper, we explore the
interplay between DNNs and game theory (GT), and show how one can benefit from
the classic readily available results from the latter when analyzing the
former. In particular, we consider the widely studied class of congestion
games, and illustrate their intrinsic relatedness to both linear and non-linear
DNNs and to the properties of their loss surface. Beyond retrieving the
state-of-the-art results from the literature, we argue that our work provides a
very promising novel tool for analyzing the DNNs and support this claim by
proposing concrete open problems that can advance significantly our
understanding of DNNs when solved.
