Unconstrained Online Optimization: Dynamic Regret Analysis of Strongly
  Convex and Smooth Problems
  The regret bound of dynamic online learning algorithms is often expressed in
terms of the variation in the function sequence ($V_T$) and/or the path-length
of the minimizer sequence after $T$ rounds. For strongly convex and smooth
functions, , Zhang et al. establish the squared path-length of the minimizer
sequence ($C^*_{2,T}$) as a lower bound on regret. They also show that online
gradient descent (OGD) achieves this lower bound using multiple gradient
queries per round. In this paper, we focus on unconstrained online
optimization. We first show that a preconditioned variant of OGD achieves
$O(C^*_{2,T})$ with one gradient query per round. We then propose online
optimistic Newton (OON) method for the case when the first and second order
information of the function sequence is predictable. The regret bound of OON is
captured via the quartic path-length of the minimizer sequence ($C^*_{4,T}$),
which can be much smaller than $C^*_{2,T}$. We finally show that by using
multiple gradients for OGD, we can achieve an upper bound of
$O(\min\{C^*_{2,T},V_T\})$ on regret.
