From Online Optimization to PID Controllers: Mirror Descent with
  Momentum
  We study a family of first-order methods with momentum based on mirror
descent for online convex optimization, which we dub online mirror descent with
momentum (OMDM). Our algorithms include as special cases gradient descent and
exponential weights update with momentum. We provide a new and simple analysis
of momentum-based methods in a stochastic setting that yields a regret bound
that decreases as momentum increases. This immediately establishes that
momentum can help in the convergence of stochastic subgradient descent in
convex nonsmooth optimization. We showcase the robustness of our algorithm by
also providing an analysis in an adversarial setting that gives the first
non-trivial regret bounds for OMDM. Our work aims to provide a better
understanding of the benefits of momentum-based methods, which despite their
recent empirical success, is incomplete.
  Finally, we discuss how OMDM can be applied to stochastic online allocation
problems, which are central problems in computer science and operations
research. In doing so, we establish an important connection between OMDM and
popular approaches from optimal control such as PID controllers, thereby
providing regret bounds on the performance of PID controllers. The improvements
of momentum are most pronounced when the step-size is large, thereby indicating
that momentum provides a robustness to misspecification of tuning parameters.
We provide a numerical evaluation that verifies the robustness of our
algorithms.
