A Modular Analysis of Adaptive (Non-)Convex Optimization: Optimism,
  Composite Objectives, and Variational Bounds
  Recently, much work has been done on extending the scope of online learning
and incremental stochastic optimization algorithms. In this paper we contribute
to this effort in two ways: First, based on a new regret decomposition and a
generalization of Bregman divergences, we provide a self-contained, modular
analysis of the two workhorses of online learning: (general) adaptive versions
of Mirror Descent (MD) and the Follow-the-Regularized-Leader (FTRL) algorithms.
The analysis is done with extra care so as not to introduce assumptions not
needed in the proofs and allows to combine, in a straightforward way, different
algorithmic ideas (e.g., adaptivity, optimism, implicit updates) and learning
settings (e.g., strongly convex or composite objectives). This way we are able
to reprove, extend and refine a large body of the literature, while keeping the
proofs concise. The second contribution is a byproduct of this careful
analysis: We present algorithms with improved variational bounds for smooth,
composite objectives, including a new family of optimistic MD algorithms with
only one projection step per round. Furthermore, we provide a simple extension
of adaptive regret bounds to practically relevant non-convex problem settings
with essentially no extra effort.
