Multi-Layered Gradient Boosting Decision Trees
  Multi-layered representation is believed to be the key ingredient of deep
neural networks especially in cognitive tasks like computer vision. While
non-differentiable models such as gradient boosting decision trees (GBDTs) are
the dominant methods for modeling discrete or tabular data, they are hard to
incorporate with such representation learning ability. In this work, we propose
the multi-layered GBDT forest (mGBDTs), with an explicit emphasis on exploring
the ability to learn hierarchical representations by stacking several layers of
regression GBDTs as its building block. The model can be jointly trained by a
variant of target propagation across layers, without the need to derive
back-propagation nor differentiability. Experiments and visualizations
confirmed the effectiveness of the model in terms of performance and
representation learning ability.
