Boosting One-Point Derivative-Free Online Optimization via Residual
  Feedback
  Zeroth-order optimization (ZO) typically relies on two-point feedback to
estimate the unknown gradient of the objective function. Nevertheless,
two-point feedback can not be used for online optimization of time-varying
objective functions, where only a single query of the function value is
possible at each time step. In this work, we propose a new one-point feedback
method for online optimization that estimates the objective function gradient
using the residual between two feedback points at consecutive time instants.
Moreover, we develop regret bounds for ZO with residual feedback for both
convex and nonconvex online optimization problems. Specifically, for both
deterministic and stochastic problems and for both Lipschitz and smooth
objective functions, we show that using residual feedback can produce gradient
estimates with much smaller variance compared to conventional one-point
feedback methods. As a result, our regret bounds are much tighter compared to
existing regret bounds for ZO with conventional one-point feedback, which
suggests that ZO with residual feedback can better track the optimizer of
online optimization problems. Additionally, our regret bounds rely on weaker
assumptions than those used in conventional one-point feedback methods.
Numerical experiments show that ZO with residual feedback significantly
outperforms existing one-point feedback methods also in practice.
