Predicting with Proxies: Transfer Learning in High Dimension
  Predictive analytics is increasingly used to guide decision-making in many
applications. However, in practice, we often have limited data on the true
predictive task of interest, and must instead rely on more abundant data on a
closely-related proxy predictive task. For example, e-commerce platforms use
abundant customer click data (proxy) to make product recommendations rather
than the relatively sparse customer purchase data (true outcome of interest);
alternatively, hospitals often rely on medical risk scores trained on a
different patient population (proxy) rather than their own patient population
(true cohort of interest) to assign interventions. Yet, not accounting for the
bias in the proxy can lead to sub-optimal decisions. Using real datasets, we
find that this bias can often be captured by a sparse function of the features.
Thus, we propose a novel two-step estimator that uses techniques from
high-dimensional statistics to efficiently combine a large amount of proxy data
and a small amount of true data. We prove upper bounds on the error of our
proposed estimator and lower bounds on several heuristics used by data
scientists; in particular, our proposed estimator can achieve the same accuracy
with exponentially less true data (in the number of features). Our proof relies
on a new LASSO tail inequality for approximately sparse vectors. Finally, we
demonstrate the effectiveness of our approach on e-commerce and healthcare
datasets; in both cases, we achieve significantly better predictive accuracy as
well as managerial insights into the nature of the bias in the proxy data.
