Online Learning with Continuous Variations: Dynamic Regret and
  Reductions
  Online learning is a powerful tool for analyzing iterative algorithms.
However, the classic adversarial setup sometimes fails to capture certain
regularity in online problems in practice. Motivated by this, we establish a
new setup, called Continuous Online Learning (COL), where the gradient of
online loss function changes continuously across rounds with respect to the
learner's decisions. We show that COL covers and more appropriately describes
many interesting applications, from general equilibrium problems (EPs) to
optimization in episodic MDPs. In particular, we show monotone EPs admits a
reduction to achieving sublinear static regret in COL. Using this new setup, we
revisit the difficulty of sublinear dynamic regret. We prove a fundamental
equivalence between achieving sublinear dynamic regret in COL and solving
certain EPs. With this insight, we offer conditions for efficient algorithms
that achieve sublinear dynamic regret, even when the losses are chosen
adaptively without any a priori variation budget. Furthermore, we show for COL
a reduction from dynamic regret to both static regret and convergence in the
associated EP, allowing us to analyze the dynamic regret of many existing
algorithms.
