Born-Infeld (BI) for AI: Energy-Conserving Descent (ECD) for
  Optimization
  We introduce a novel framework for optimization based on energy-conserving
Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish its
key properties analytically and numerically. The prototype is a discretization
of Born-Infeld dynamics, with a squared relativistic speed limit depending on
the objective function. This class of frictionless, energy-conserving
optimizers proceeds unobstructed until slowing naturally near the minimal loss,
which dominates the phase space volume of the system. Building from studies of
chaotic systems such as dynamical billiards, we formulate a specific algorithm
with good performance on machine learning and PDE-solving tasks, including
generalization. It cannot stop at a high local minimum, an advantage in
non-convex loss functions, and proceeds faster than GD+momentum in shallow
valleys.
