More Powerful Selective Kernel Tests for Feature Selection
  Refining one's hypotheses in the light of data is a common scientific
practice; however, the dependency on the data introduces selection bias and can
lead to specious statistical analysis. An approach for addressing this is via
conditioning on the selection procedure to account for how we have used the
data to generate our hypotheses, and prevent information to be used again after
selection. Many selective inference (a.k.a. post-selection inference)
algorithms typically take this approach but will "over-condition" for sake of
tractability. While this practice yields well calibrated statistic tests with
controlled false positive rates (FPR), it can incur a major loss in power. In
our work, we extend two recent proposals for selecting features using the
Maximum Mean Discrepancy and Hilbert Schmidt Independence Criterion to
condition on the minimal conditioning event. We show how recent advances in
multiscale bootstrap makes conditioning on the minimal selection event possible
and demonstrate our proposal over a range of synthetic and real world
experiments. Our results show that our proposed test is indeed more powerful in
most scenarios.
