Constants of Motion: The Antidote to Chaos in Optimization and Game
  Dynamics
  Several recent works in online optimization and game dynamics have
established strong negative complexity results including the formal emergence
of instability and chaos even in small such settings, e.g., $2\times 2$ games.
These results motivate the following question: Which methodological tools can
guarantee the regularity of such dynamics and how can we apply them in standard
settings of interest such as discrete-time first-order optimization dynamics?
We show how proving the existence of invariant functions, i.e., constant of
motions, is a fundamental contribution in this direction and establish a
plethora of such positive results (e.g. gradient descent, multiplicative
weights update, alternating gradient descent and manifold gradient descent)
both in optimization as well as in game settings. At a technical level, for
some conservation laws we provide an explicit and concise closed form, whereas
for other ones we present non-constructive proofs using tools from dynamical
systems.
