Merging versus Ensembling in Multi-Study Prediction: Theoretical Insight
  from Random Effects
  A critical decision point when training predictors using multiple studies is
whether these studies should be combined or treated separately. We compare two
multi-study learning approaches in the presence of potential heterogeneity in
predictor-outcome relationships across datasets. We consider 1) merging all of
the datasets and training a single learner, and 2) multi-study ensembling,
which involves training a separate learner on each dataset and combining the
predictions resulting from each learner. In a linear regression setting, we
show analytically and confirm via simulation that merging yields lower
prediction error than ensembling when the predictor-outcome relationships are
relatively homogeneous across studies. However, as cross-study heterogeneity
increases, there exists a transition point beyond which ensembling outperforms
merging. We provide analytic expressions for the transition point in various
scenarios, study asymptotic properties, and illustrate how transition point
theory can be used for deciding when studies should be combined with an
application from metabolomics.
