GPUTreeShap: Massively Parallel Exact Calculation of SHAP Scores for
  Tree Ensembles
  SHAP (SHapley Additive exPlanation) values provide a game theoretic
interpretation of the predictions of machine learning models based on Shapley
values. While exact calculation of SHAP values is computationally intractable
in general, a recursive polynomial-time algorithm called TreeShap is available
for decision tree models. However, despite its polynomial time complexity,
TreeShap can become a significant bottleneck in practical machine learning
pipelines when applied to large decision tree ensembles. Unfortunately, the
complicated TreeShap algorithm is difficult to map to hardware accelerators
such as GPUs. In this work, we present GPUTreeShap, a reformulated TreeShap
algorithm suitable for massively parallel computation on graphics processing
units. Our approach first preprocesses each decision tree to isolate variable
sized sub-problems from the original recursive algorithm, then solves a bin
packing problem, and finally maps sub-problems to single-instruction,
multiple-thread (SIMT) tasks for parallel execution with specialised hardware
instructions. With a single NVIDIA Tesla V100-32 GPU, we achieve speedups of up
to 19x for SHAP values, and speedups of up to 340x for SHAP interaction values,
over a state-of-the-art multi-core CPU implementation executed on two 20-core
Xeon E5-2698 v4 2.2 GHz CPUs. We also experiment with multi-GPU computing using
eight V100 GPUs, demonstrating throughput of 1.2M rows per second -- equivalent
CPU-based performance is estimated to require 6850 CPU cores.
