A Numerical Transform of Random Forest Regressors corrects
  Systematically-Biased Predictions
  Over the past decade, random forest models have become widely used as a
robust method for high-dimensional data regression tasks. In part, the
popularity of these models arises from the fact that they require little
hyperparameter tuning and are not very susceptible to overfitting. Random
forest regression models are comprised of an ensemble of decision trees that
independently predict the value of a (continuous) dependent variable;
predictions from each of the trees are ultimately averaged to yield an overall
predicted value from the forest. Using a suite of representative real-world
datasets, we find a systematic bias in predictions from random forest models.
We find that this bias is recapitulated in simple synthetic datasets,
regardless of whether or not they include irreducible error (noise) in the
data, but that models employing boosting do not exhibit this bias. Here we
demonstrate the basis for this problem, and we use the training data to define
a numerical transformation that fully corrects it. Application of this
transformation yields improved predictions in every one of the real-world and
synthetic datasets evaluated in our study.
