Low Rank Regularization: A Review
  Low rank regularization, in essence, involves introducing a low rank or
approximately low rank assumption for matrix we aim to learn, which has
achieved great success in many fields including machine learning, data mining
and computer version. Over the last decade, much progress has been made in
theories and practical applications. Nevertheless, the intersection between
them is very slight. In order to construct a bridge between practical
applications and theoretical research, in this paper we provide a comprehensive
survey for low rank regularization. We first review several traditional machine
learning models using low rank regularization, and then show their (or their
variants) applications in solving practical issues, such as non-rigid structure
from motion and image denoising. Subsequently, we summarize the regularizers
and optimization methods that achieve great success in traditional machine
learning tasks but are rarely seen in solving practical issues. Finally, we
provide a discussion and comparison for some representative regularizers
including convex and non-convex relaxations. Extensive experimental results
demonstrate that non-convex regularizers can provide a large advantage over the
nuclear norm, the regularizer widely used in solving practical issues.
