A Unified Framework for Sparse Relaxed Regularized Regression: SR3
  Regularized regression problems are ubiquitous in statistical modeling,
signal processing, and machine learning. Sparse regression in particular has
been instrumental in scientific model discovery, including compressed sensing
applications, variable selection, and high-dimensional analysis. We propose a
broad framework for sparse relaxed regularized regression, called SR3. The key
idea is to solve a relaxation of the regularized problem, which has three
advantages over the state-of-the-art: (1) solutions of the relaxed problem are
superior with respect to errors, false positives, and conditioning, (2)
relaxation allows extremely fast algorithms for both convex and nonconvex
formulations, and (3) the methods apply to composite regularizers such as total
variation (TV) and its nonconvex variants. We demonstrate the advantages of SR3
(computational efficiency, higher accuracy, faster convergence rates, greater
flexibility) across a range of regularized regression problems with synthetic
and real data, including applications in compressed sensing, LASSO, matrix
completion, TV regularization, and group sparsity. To promote reproducible
research, we also provide a companion MATLAB package that implements these
examples.
