Explore Aggressively, Update Conservatively: Stochastic Extragradient
  Methods with Variable Stepsize Scaling
  Owing to their stability and convergence speed, extragradient methods have
become a staple for solving large-scale saddle-point problems in machine
learning. The basic premise of these algorithms is the use of an extrapolation
step before performing an update; thanks to this exploration step,
extra-gradient methods overcome many of the non-convergence issues that plague
gradient descent/ascent schemes. On the other hand, as we show in this paper,
running vanilla extragradient with stochastic gradients may jeopardize its
convergence, even in simple bilinear models. To overcome this failure, we
investigate a double stepsize extragradient algorithm where the exploration
step evolves at a more aggressive time-scale compared to the update step. We
show that this modification allows the method to converge even with stochastic
gradients, and we derive sharp convergence rates under an error bound
condition.
