A Stochastic Halpern Iteration with Variance Reduction for Stochastic
  Monotone Inclusion Problems
  We study stochastic monotone inclusion problems, which widely appear in
machine learning applications, including robust regression and adversarial
learning. We propose novel variants of stochastic Halpern iteration with
recursive variance reduction. In the cocoercive -- and more generally
Lipschitz-monotone -- setup, our algorithm attains $\epsilon$ norm of the
operator with $\mathcal{O}(\frac{1}{\epsilon^3})$ stochastic operator
evaluations, which significantly improves over state of the art
$\mathcal{O}(\frac{1}{\epsilon^4})$ stochastic operator evaluations required
for existing monotone inclusion solvers applied to the same problem classes. We
further show how to couple one of the proposed variants of stochastic Halpern
iteration with a scheduled restart scheme to solve stochastic monotone
inclusion problems with ${\mathcal{O}}(\frac{\log(1/\epsilon)}{\epsilon^2})$
stochastic operator evaluations under additional sharpness or strong
monotonicity assumptions. Finally, we argue via reductions between different
problem classes that our stochastic oracle complexity bounds are tight up to
logarithmic factors in terms of their $\epsilon$-dependence.
