Estimate Sequences for Variance-Reduced Stochastic Composite
  Optimization
  In this paper, we propose a unified view of gradient-based algorithms for
stochastic convex composite optimization by extending the concept of estimate
sequence introduced by Nesterov. This point of view covers the stochastic
gradient descent method, variants of the approaches SAGA, SVRG, and has several
advantages: (i) we provide a generic proof of convergence for the
aforementioned methods; (ii) we show that this SVRG variant is adaptive to
strong convexity; (iii) we naturally obtain new algorithms with the same
guarantees; (iv) we derive generic strategies to make these algorithms robust
to stochastic noise, which is useful when data is corrupted by small random
perturbations. Finally, we show that this viewpoint is useful to obtain new
accelerated algorithms in the sense of Nesterov.
