Nonconvex Factorization and Manifold Formulations are Almost Equivalent
  in Low-rank Matrix Optimization
  In this paper, we consider the geometric landscape connection of the widely
studied manifold and factorization formulations in low-rank positive
semidefinite (PSD) and general matrix optimization. We establish an equivalence
on the set of first-order stationary points (FOSPs) and second-order stationary
points (SOSPs) between the manifold and the factorization formulations. We
further give a sandwich inequality on the spectrum of Riemannian and Euclidean
Hessians at FOSPs, which can be used to transfer more geometric properties from
one formulation to another. Similarities and differences on the landscape
connection under the PSD case and the general case are discussed. To the best
of our knowledge, this is the first geometric landscape connection between the
manifold and the factorization formulations for handling rank constraints. In
the general low-rank matrix optimization, the landscape connection of two
factorization formulations (unregularized and regularized ones) is also
provided. By applying these geometric landscape connections, we are able to
solve unanswered questions in literature and establish stronger results in the
applications on geometric analysis of phase retrieval, well-conditioned
low-rank matrix optimization, and the role of regularization in factorization
arising from machine learning and signal processing.
