Sparse learning with CART
  Decision trees with binary splits are popularly constructed using
Classification and Regression Trees (CART) methodology. For regression models,
this approach recursively divides the data into two near-homogenous daughter
nodes according to a split point that maximizes the reduction in sum of squares
error (the impurity) along a particular variable. This paper aims to study the
statistical properties of regression trees constructed with CART methodology.
In doing so, we find that the training error is governed by the Pearson
correlation between the optimal decision stump and response data in each node,
which we bound by constructing a prior distribution on the split points and
solving a nonlinear optimization problem. We leverage this connection between
the training error and Pearson correlation to show that CART with
cost-complexity pruning achieves an optimal complexity/goodness-of-fit tradeoff
when the depth scales with the logarithm of the sample size. Data dependent
quantities, which adapt to the dimensionality and latent structure of the
regression model, are seen to govern the rates of convergence of the prediction
error.
