Logarithmic Regret for Online Gradient Descent Beyond Strong Convexity
  Hoffman's classical result gives a bound on the distance of a point from a
convex and compact polytope in terms of the magnitude of violation of the
constraints. Recently, several results showed that Hoffman's bound can be used
to derive strongly-convex-like rates for first-order methods for
\textit{offline} convex optimization of curved, though not strongly convex,
functions, over polyhedral sets. In this work, we use this classical result for
the first time to obtain faster rates for \textit{online convex optimization}
over polyhedral sets with curved convex, though not strongly convex, loss
functions. We show that under several reasonable assumptions on the data, the
standard \textit{Online Gradient Descent} algorithm guarantees logarithmic
regret. To the best of our knowledge, the only previous algorithm to achieve
logarithmic regret in the considered settings is the \textit{Online Newton
Step} algorithm which requires quadratic (in the dimension) memory and at least
quadratic runtime per iteration, which greatly limits its applicability to
large-scale problems. In particular, our results hold for
\textit{semi-adversarial} settings in which the data is a combination of an
arbitrary (adversarial) sequence and a stochastic sequence, which might provide
reasonable approximation for many real-world sequences, or under a natural
assumption that the data is low-rank. We demonstrate via experiments that the
regret of OGD is indeed comparable to that of ONS (and even far better) on
curved though not strongly-convex losses.
