Parametric Programming Approach for More Powerful and General Lasso
  Selective Inference
  Selective Inference (SI) has been actively studied in the past few years for
conducting inference on the features of linear models that are adaptively
selected by feature selection methods such as Lasso. The basic idea of SI is to
make inference conditional on the selection event. Unfortunately, the main
limitation of the original SI approach for Lasso is that the inference is
conducted not only conditional on the selected features but also on their signs
-- this leads to loss of power because of over-conditioning. Although this
limitation can be circumvented by considering the union of such selection
events for all possible combinations of signs, this is only feasible when the
number of selected features is sufficiently small. To address this
computational bottleneck, we propose a parametric programming-based method that
can conduct SI without conditioning on signs even when we have thousands of
active features. The main idea is to compute the continuum path of Lasso
solutions in the direction of a test statistic, and identify the subset of the
data space corresponding to the feature selection event by following the
solution path. The proposed parametric programming-based method not only avoids
the aforementioned computational bottleneck but also improves the performance
and practicality of SI for Lasso in various respects. We conduct several
experiments to demonstrate the effectiveness and efficiency of our proposed
method.
