Universal Consistency of Decision Trees in High Dimensions
  This paper shows that decision trees constructed with Classification and
Regression Trees (CART) methodology are universally consistent in an additive
model context, even when the number of predictor variables scales exponentially
with the sample size, under certain $1$-norm sparsity constraints. The
consistency is universal in the sense that there are no a priori assumptions on
the distribution of the predictor variables. Amazingly, this adaptivity to
(approximate or exact) sparsity is achieved with a single tree, as opposed to
what might be expected for an ensemble. Finally, we show that these qualitative
properties of individual trees are inherited by Breiman's random forests.
Another surprise is that consistency holds even when the "mtry" tuning
parameter vanishes as a fraction of the number of predictor variables, thus
speeding up computation of the forest. A key step in the analysis is the
establishment of an oracle inequality, which precisely characterizes the
goodness-of-fit and complexity tradeoff for a misspecified model.
