On an improvement of LASSO by scaling
  A sparse modeling is a major topic in machine learning and statistics. LASSO
(Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling
method while it has been known to yield unexpected large bias especially at a
sparse representation. There have been several studies for improving this
problem such as the introduction of non-convex regularization terms. The
important point is that this bias problem directly affects model selection in
applications since a sparse representation cannot be selected by a prediction
error based model selection even if it is a good representation. In this
article, we considered to improve this problem by introducing a scaling that
expands LASSO estimator to compensate excessive shrinkage, thus a large bias in
LASSO estimator. We here gave an empirical value for the amount of scaling.
There are two advantages of this scaling method as follows. Since the proposed
scaling value is calculated by using LASSO estimator, we only need LASSO
estimator that is obtained by a fast and stable optimization procedure such as
LARS (Least Angle Regression) under LASSO modification or coordinate descent.
And, the simplicity of our scaling method enables us to derive SURE (Stein's
Unbiased Risk Estimate) under the modified LASSO estimator with scaling. Our
scaling method together with model selection based on SURE is fully empirical
and do not need additional hyper-parameters. In a simple numerical example, we
verified that our scaling method actually improves LASSO and the SURE based
model selection criterion can stably choose an appropriate sparse model.
