Feedback-Controlled Sequential Lasso Screening
  One way to solve lasso problems when the dictionary does not fit into
available memory is to first screen the dictionary to remove unneeded features.
Prior research has shown that sequential screening methods offer the greatest
promise in this endeavor. Most existing work on sequential screening targets
the context of tuning parameter selection, where one screens and solves a
sequence of $N$ lasso problems with a fixed grid of geometrically spaced
regularization parameters. In contrast, we focus on the scenario where a target
regularization parameter has already been chosen via cross-validated model
selection, and we then need to solve many lasso instances using this fixed
value. In this context, we propose and explore a feedback controlled sequential
screening scheme. Feedback is used at each iteration to select the next problem
to be solved. This allows the sequence of problems to be adapted to the
instance presented and the number of intermediate problems to be automatically
selected. We demonstrate our feedback scheme using several datasets including a
dictionary of approximate size 100,000 by 300,000.
