MT-GBM: A Multi-Task Gradient Boosting Machine with Shared Decision
  Trees
  Despite the success of deep learning in computer vision and natural language
processing, Gradient Boosted Decision Tree (GBDT) is yet one of the most
powerful tools for applications with tabular data such as e-commerce and
FinTech. However, applying GBDT to multi-task learning is still a challenge.
Unlike deep models that can jointly learn a shared latent representation across
multiple tasks, GBDT can hardly learn a shared tree structure. In this paper,
we propose Multi-task Gradient Boosting Machine (MT-GBM), a GBDT-based method
for multi-task learning. The MT-GBM can find the shared tree structures and
split branches according to multi-task losses. First, it assigns multiple
outputs to each leaf node. Next, it computes the gradient corresponding to each
output (task). Then, we also propose an algorithm to combine the gradients of
all tasks and update the tree. Finally, we apply MT-GBM to LightGBM.
Experiments show that our MT-GBM improves the performance of the main task
significantly, which means the proposed MT-GBM is efficient and effective.
