Feature-Weighted Linear Stacking
  Ensemble methods, such as stacking, are designed to boost predictive accuracy
by blending the predictions of multiple machine learning models. Recent work
has shown that the use of meta-features, additional inputs describing each
example in a dataset, can boost the performance of ensemble methods, but the
greatest reported gains have come from nonlinear procedures requiring
significant tuning and training time. Here, we present a linear technique,
Feature-Weighted Linear Stacking (FWLS), that incorporates meta-features for
improved accuracy while retaining the well-known virtues of linear regression
regarding speed, stability, and interpretability. FWLS combines model
predictions linearly using coefficients that are themselves linear functions of
meta-features. This technique was a key facet of the solution of the second
place team in the recently concluded Netflix Prize competition. Significant
increases in accuracy over standard linear stacking are demonstrated on the
Netflix Prize collaborative filtering dataset.
