KTBoost: Combined Kernel and Tree Boosting
  We introduce a novel boosting algorithm called `KTBoost' which combines
kernel boosting and tree boosting. In each boosting iteration, the algorithm
adds either a regression tree or reproducing kernel Hilbert space (RKHS)
regression function to the ensemble of base learners. Intuitively, the idea is
that discontinuous trees and continuous RKHS regression functions complement
each other, and that this combination allows for better learning of functions
that have parts with varying degrees of regularity such as discontinuities and
smooth parts. We empirically show that KTBoost significantly outperforms both
tree and kernel boosting in terms of predictive accuracy in a comparison on a
wide array of data sets.
