A Discrete Variational Derivation of Accelerated Methods in Optimization
  Many of the new developments in machine learning are connected with
gradient-based optimization methods. Recently, these methods have been studied
using a variational perspective. This has opened up the possibility of
introducing variational and symplectic methods using geometric integration. In
particular, in this paper, we introduce variational integrators which allow us
to derive different methods for optimization. Using both, Hamilton's and
Lagrange-d'Alembert's principle, we derive two families of respective
optimization methods in one-to-one correspondence that generalize Polyak's
heavy ball and the well known Nesterov accelerated gradient method, the second
of which mimics the behavior of the first reducing the oscillations of
classical momentum methods. However, since the systems considered are
explicitly time-dependent, the preservation of symplecticity of autonomous
systems occurs here solely on the fibers. Several experiments exemplify the
result.
