Slow-Growing Trees
  Random Forest's performance can be matched by a single slow-growing tree
(SGT), which uses a learning rate to tame CART's greedy algorithm. SGT exploits
the view that CART is an extreme case of an iterative weighted least square
procedure. Moreover, a unifying view of Boosted Trees (BT) and Random Forests
(RF) is presented. Greedy ML algorithms' outcomes can be improved using either
"slow learning" or diversification. SGT applies the former to estimate a single
deep tree, and Booging (bagging stochastic BT with a high learning rate) uses
the latter with additive shallow trees. The performance of this tree ensemble
quaternity (Booging, BT, SGT, RF) is assessed on simulated and real regression
tasks.
