Dual Adaptivity: A Universal Algorithm for Minimizing the Adaptive
  Regret of Convex Functions
  To deal with changing environments, a new performance measure -- adaptive
regret, defined as the maximum static regret over any interval, was proposed in
online learning. Under the setting of online convex optimization, several
algorithms have been successfully developed to minimize the adaptive regret.
However, existing algorithms lack universality in the sense that they can only
handle one type of convex functions and need apriori knowledge of parameters.
By contrast, there exist universal algorithms, such as MetaGrad, that attain
optimal static regret for multiple types of convex functions simultaneously.
Along this line of research, this paper presents the first universal algorithm
for minimizing the adaptive regret of convex functions. Specifically, we borrow
the idea of maintaining multiple learning rates in MetaGrad to handle the
uncertainty of functions, and utilize the technique of sleeping experts to
capture changing environments. In this way, our algorithm automatically adapts
to the property of functions (convex, exponentially concave, or strongly
convex), as well as the nature of environments (stationary or changing). As a
by product, it also allows the type of functions to switch between rounds.
