Conditional Uncorrelation and Efficient Non-approximate Subset Selection
  in Sparse Regression
  Given $m$ $d$-dimensional responsors and $n$ $d$-dimensional predictors,
sparse regression finds at most $k$ predictors for each responsor for linear
approximation, $1\leq k \leq d-1$. The key problem in sparse regression is
subset selection, which usually suffers from high computational cost. Recent
years, many improved approximate methods of subset selection have been
published. However, less attention has been paid on the non-approximate method
of subset selection, which is very necessary for many questions in data
analysis. Here we consider sparse regression from the view of correlation, and
propose the formula of conditional uncorrelation. Then an efficient
non-approximate method of subset selection is proposed in which we do not need
to calculate any coefficients in regression equation for candidate predictors.
By the proposed method, the computational complexity is reduced from
$O(\frac{1}{6}{k^3}+mk^2+mkd)$ to $O(\frac{1}{6}{k^3}+\frac{1}{2}mk^2)$ for
each candidate subset in sparse regression. Because the dimension $d$ is
generally the number of observations or experiments and large enough, the
proposed method can greatly improve the efficiency of non-approximate subset
selection.
