Projection-free nonconvex stochastic optimization on Riemannian
  manifolds
  We study stochastic projection-free methods for constrained optimization of
smooth functions on Riemannian manifolds, i.e., with additional constraints
beyond the parameter domain being a manifold. Specifically, we introduce
stochastic Riemannian Frank-Wolfe methods for nonconvex and geodesically convex
problems. We present algorithms for both purely stochastic optimization and
finite-sum problems. For the latter, we develop variance-reduced methods,
including a Riemannian adaptation of the recently proposed Spider technique.
For all settings, we recover convergence rates that are comparable to the
best-known rates for their Euclidean counterparts. Finally, we discuss
applications to two classic tasks: The computation of the Karcher mean of
positive definite matrices and Wasserstein barycenters for multivariate normal
distributions. For both tasks, stochastic Fw methods yield state-of-the-art
empirical performance.
