Balance is key: Private median splits yield high-utility random trees
  Random forests are a popular method for classification and regression due to
their versatility. However, this flexibility can come at the cost of user
privacy, since training random forests requires multiple data queries, often on
small, identifiable subsets of the training data. Privatizing these queries
typically comes at a high utility cost, in large part because we are
privatizing queries on small subsets of the data, which are easily corrupted by
added noise. In this paper, we propose DiPriMe forests, a novel tree-based
ensemble method for differentially private regression and classification, which
is appropriate for real or categorical covariates. We generate splits using a
differentially private version of the median, which encourages balanced leaf
nodes. By avoiding low occupancy leaf nodes, we avoid high signal-to-noise
ratios when privatizing the leaf node sufficient statistics. We show
theoretically and empirically that the resulting algorithm exhibits high
utility, while ensuring differential privacy.
