Accelerated Stochastic Quasi-Newton Optimization on Riemann Manifolds
  We propose an L-BFGS optimization algorithm on Riemannian manifolds using
minibatched stochastic variance reduction techniques for fast convergence with
constant step sizes, without resorting to linesearch methods designed to
satisfy Wolfe conditions. We provide a new convergence proof for strongly
convex functions without using curvature conditions on the manifold, as well as
a convergence discussion for nonconvex functions. We discuss a couple of ways
to obtain the correction pairs used to calculate the product of the gradient
with the inverse Hessian, and empirically demonstrate their use in synthetic
experiments on computation of Karcher means for symmetric positive definite
matrices and leading eigenvalues of large scale data matrices. We compare our
method to VR-PCA for the latter experiment, along with Riemannian SVRG for both
cases, and show strong convergence results for a range of datasets.
