Fast Low-Rank Matrix Learning with Nonconvex Regularization
  Low-rank modeling has a lot of important applications in machine learning,
computer vision and social network analysis. While the matrix rank is often
approximated by the convex nuclear norm, the use of nonconvex low-rank
regularizers has demonstrated better recovery performance. However, the
resultant optimization problem is much more challenging. A very recent
state-of-the-art is based on the proximal gradient algorithm. However, it
requires an expensive full SVD in each proximal step. In this paper, we show
that for many commonly-used nonconvex low-rank regularizers, a cutoff can be
derived to automatically threshold the singular values obtained from the
proximal operator. This allows the use of power method to approximate the SVD
efficiently. Besides, the proximal operator can be reduced to that of a much
smaller matrix projected onto this leading subspace. Convergence, with a rate
of O(1/T) where T is the number of iterations, can be guaranteed. Extensive
experiments are performed on matrix completion and robust principal component
analysis. The proposed method achieves significant speedup over the
state-of-the-art. Moreover, the matrix solution obtained is more accurate and
has a lower rank than that of the traditional nuclear norm regularizer.
