Asymptotic Escape of Spurious Critical Points on the Low-rank Matrix
  Manifold
  We show that on the manifold of fixed-rank and symmetric positive
semi-definite matrices, the Riemannian gradient descent algorithm almost surely
escapes some spurious critical points on the boundary of the manifold. Our
result is the first to partially overcome the incompleteness of the low-rank
matrix manifold without changing the vanilla Riemannian gradient descent
algorithm. The spurious critical points are some rank-deficient matrices that
capture only part of the eigen components of the ground truth. Unlike classical
strict saddle points, they exhibit very singular behavior. We show that using
the dynamical low-rank approximation and a rescaled gradient flow, some of the
spurious critical points can be converted to classical strict saddle points in
the parameterized domain, which leads to the desired result. Numerical
experiments are provided to support our theoretical findings.
