On the Convergence of Stochastic Extragradient for Bilinear Games using
  Restarted Iteration Averaging
  We study the stochastic bilinear minimax optimization problem, presenting an
analysis of the same-sample Stochastic ExtraGradient (SEG) method with constant
step size, and presenting variations of the method that yield favorable
convergence. In sharp contrasts with the basic SEG method whose last iterate
only contracts to a fixed neighborhood of the Nash equilibrium, SEG augmented
with iteration averaging provably converges to the Nash equilibrium under the
same standard settings, and such a rate is further improved by incorporating a
scheduled restarting procedure. In the interpolation setting where noise
vanishes at the Nash equilibrium, we achieve an optimal convergence rate up to
tight constants. We present numerical experiments that validate our theoretical
findings and demonstrate the effectiveness of the SEG method when equipped with
iteration averaging and restarting.
