A Survey on Nonconvex Regularization Based Sparse and Low-Rank Recovery
  in Signal Processing, Statistics, and Machine Learning
  In the past decade, sparse and low-rank recovery have drawn much attention in
many areas such as signal/image processing, statistics, bioinformatics and
machine learning. To achieve sparsity and/or low-rankness inducing, the
$\ell_1$ norm and nuclear norm are of the most popular regularization penalties
due to their convexity. While the $\ell_1$ and nuclear norm are convenient as
the related convex optimization problems are usually tractable, it has been
shown in many applications that a nonconvex penalty can yield significantly
better performance. In recent, nonconvex regularization based sparse and
low-rank recovery is of considerable interest and it in fact is a main driver
of the recent progress in nonconvex and nonsmooth optimization. This paper
gives an overview of this topic in various fields in signal processing,
statistics and machine learning, including compressive sensing (CS), sparse
regression and variable selection, sparse signals separation, sparse principal
component analysis (PCA), large covariance and inverse covariance matrices
estimation, matrix completion, and robust PCA. We present recent developments
of nonconvex regularization based sparse and low-rank recovery in these fields,
addressing the issues of penalty selection, applications and the convergence of
nonconvex algorithms. Code is available at https://github.com/FWen/ncreg.git.
