Isotuning With Applications To Scale-Free Online Learning
  We extend and combine several tools of the literature to design fast,
adaptive, anytime and scale-free online learning algorithms. Scale-free regret
bounds must scale linearly with the maximum loss, both toward large losses and
toward very small losses. Adaptive regret bounds demonstrate that an algorithm
can take advantage of easy data and potentially have constant regret. We seek
to develop fast algorithms that depend on as few parameters as possible, in
particular they should be anytime and thus not depend on the time horizon. Our
first and main tool, isotuning, is a generalization of the idea of balancing
the trade-off of the regret. We develop a set of tools to design and analyze
such learning rates easily and show that they adapts automatically to the rate
of the regret (whether constant, $O(\log T)$, $O(\sqrt{T})$, etc.) within a
factor 2 of the optimal learning rate in hindsight for the same observed
quantities. The second tool is an online correction, which allows us to obtain
centered bounds for many algorithms, to prevent the regret bounds from being
vacuous when the domain is overly large or only partially constrained. The last
tool, null updates, prevents the algorithm from performing overly large
updates, which could result in unbounded regret, or even invalid updates. We
develop a general theory using these tools and apply it to several standard
algorithms. In particular, we (almost entirely) restore the adaptivity to small
losses of FTRL for unbounded domains, design and prove scale-free adaptive
guarantees for a variant of Mirror Descent (at least when the Bregman
divergence is convex in its second argument), extend Adapt-ML-Prod to
scale-free guarantees, and provide several other minor contributions about
Prod, AdaHedge, BOA and Soft-Bayes.
