A Simple yet Universal Strategy for Online Convex Optimization
  Recently, several universal methods have been proposed for online convex
optimization, and attain minimax rates for multiple types of convex functions
simultaneously. However, they need to design and optimize one surrogate loss
for each type of functions, which makes it difficult to exploit the structure
of the problem and utilize the vast amount of existing algorithms. In this
paper, we propose a simple strategy for universal online convex optimization,
which avoids these limitations. The key idea is to construct a set of experts
to process the original online functions, and deploy a meta-algorithm over the
\emph{linearized} losses to aggregate predictions from experts. Specifically,
we choose Adapt-ML-Prod to track the best expert, because it has a second-order
bound and can be used to leverage strong convexity and exponential concavity.
In this way, we can plug in off-the-shelf online solvers as black-box experts
to deliver problem-dependent regret bounds. Furthermore, our strategy inherits
the theoretical guarantee of any expert designed for strongly convex functions
and exponentially concave functions, up to a double logarithmic factor. For
general convex functions, it maintains the minimax optimality and also achieves
a small-loss bound.
