Nonparametric Variable Screening with Optimal Decision Stumps
  Decision trees and their ensembles are endowed with a rich set of diagnostic
tools for ranking and screening variables in a predictive model. Despite the
widespread use of tree based variable importance measures, pinning down their
theoretical properties has been challenging and therefore largely unexplored.
To address this gap between theory and practice, we derive finite sample
performance guarantees for variable selection in nonparametric models using a
single-level CART decision tree (a decision stump). Under standard operating
assumptions in variable screening literature, we find that the marginal signal
strength of each variable and ambient dimensionality can be considerably weaker
and higher, respectively, than state-of-the-art nonparametric variable
selection methods. Furthermore, unlike previous marginal screening methods that
attempt to directly estimate each marginal projection via a truncated basis
expansion, the fitted model used here is a simple, parsimonious decision stump,
thereby eliminating the need for tuning the number of basis terms. Thus,
surprisingly, even though decision stumps are highly inaccurate for estimation
purposes, they can still be used to perform consistent model selection.
