What is being transferred in transfer learning?
  One desired capability for machines is the ability to transfer their
knowledge of one domain to another where data is (usually) scarce. Despite
ample adaptation of transfer learning in various deep learning applications, we
yet do not understand what enables a successful transfer and which part of the
network is responsible for that. In this paper, we provide new tools and
analyses to address these fundamental questions. Through a series of analyses
on transferring to block-shuffled images, we separate the effect of feature
reuse from learning low-level statistics of data and show that some benefit of
transfer learning comes from the latter. We present that when training from
pre-trained weights, the model stays in the same basin in the loss landscape
and different instances of such model are similar in feature space and close in
parameter space.
