Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex
  Optimization
  The problem of minimizing sum-of-nonconvex functions (i.e., convex functions
that are average of non-convex ones) is becoming increasingly important in
machine learning, and is the core machinery for PCA, SVD, regularized Newton's
method, accelerated non-convex optimization, and more.
  We show how to provably obtain an accelerated stochastic algorithm for
minimizing sum-of-nonconvex functions, by $\textit{adding one additional line}$
to the well-known SVRG method. This line corresponds to momentum, and shows how
to directly apply momentum to the finite-sum stochastic minimization of
sum-of-nonconvex functions. As a side result, our method enjoys linear parallel
speed-up using mini-batch.
