Variable Grouping Based Bayesian Additive Regression Tree
  Using ensemble methods for regression has been a large success in obtaining
high-accuracy prediction. Examples are Bagging, Random forest, Boosting, BART
(Bayesian additive regression tree), and their variants. In this paper, we
propose a new perspective named variable grouping to enhance the predictive
performance. The main idea is to seek for potential grouping of variables in
such way that there is no nonlinear interaction term between variables of
different groups. Given a sum-of-learner model, each learner will only be
responsible for one group of variables, which would be more efficient in
modeling nonlinear interactions. We propose a two-stage method named variable
grouping based Bayesian additive regression tree (GBART) with a well-developed
python package gbart available. The first stage is to search for potential
interactions and an appropriate grouping of variables. The second stage is to
build a final model based on the discovered groups. Experiments on synthetic
and real data show that the proposed method can perform significantly better
than classical approaches.
