Adaptive Histogram-Based Gradient Boosted Trees for Federated Learning
  Federated Learning (FL) is an approach to collaboratively train a model
across multiple parties without sharing data between parties or an aggregator.
It is used both in the consumer domain to protect personal data as well as in
enterprise settings, where dealing with data domicile regulation and the
pragmatics of data silos are the main drivers. While gradient boosted tree
implementations such as XGBoost have been very successful for many use cases,
its federated learning adaptations tend to be very slow due to using
cryptographic and privacy methods and have not experienced widespread use. We
propose the Party-Adaptive XGBoost (PAX) for federated learning, a novel
implementation of gradient boosting which utilizes a party adaptive histogram
aggregation method, without the need for data encryption. It constructs a
surrogate representation of the data distribution for finding splits of the
decision tree. Our experimental results demonstrate strong model performance,
especially on non-IID distributions, and significantly faster training run-time
across different data sets than existing federated implementations. This
approach makes the use of gradient boosted trees practical in enterprise
federated learning.
