A Riemannian Block Coordinate Descent Method for Computing the
  Projection Robust Wasserstein Distance
  The Wasserstein distance has become increasingly important in machine
learning and deep learning. Despite its popularity, the Wasserstein distance is
hard to approximate because of the curse of dimensionality. A recently proposed
approach to alleviate the curse of dimensionality is to project the sampled
data from the high dimensional probability distribution onto a
lower-dimensional subspace, and then compute the Wasserstein distance between
the projected data. However, this approach requires to solve a max-min problem
over the Stiefel manifold, which is very challenging in practice. The only
existing work that solves this problem directly is the RGAS (Riemannian
Gradient Ascent with Sinkhorn Iteration) algorithm, which requires to solve an
entropy-regularized optimal transport problem in each iteration, and thus can
be costly for large-scale problems. In this paper, we propose a Riemannian
block coordinate descent (RBCD) method to solve this problem, which is based on
a novel reformulation of the regularized max-min problem over the Stiefel
manifold. We show that the complexity of arithmetic operations for RBCD to
obtain an $\epsilon$-stationary point is $O(\epsilon^{-3})$. This significantly
improves the corresponding complexity of RGAS, which is $O(\epsilon^{-12})$.
Moreover, our RBCD has very low per-iteration complexity, and hence is suitable
for large-scale problems. Numerical results on both synthetic and real datasets
demonstrate that our method is more efficient than existing methods, especially
when the number of sampled data is very large.
