Probing transfer learning with a model of synthetic correlated datasets
  Transfer learning can significantly improve the sample efficiency of neural
networks, by exploiting the relatedness between a data-scarce target task and a
data-abundant source task. Despite years of successful applications, transfer
learning practice often relies on ad-hoc solutions, while theoretical
understanding of these procedures is still limited. In the present work, we
re-think a solvable model of synthetic data as a framework for modeling
correlation between data-sets. This setup allows for an analytic
characterization of the generalization performance obtained when transferring
the learned feature map from the source to the target task. Focusing on the
problem of training two-layer networks in a binary classification setting, we
show that our model can capture a range of salient features of transfer
learning with real data. Moreover, by exploiting parametric control over the
correlation between the two data-sets, we systematically investigate under
which conditions the transfer of features is beneficial for generalization.
