Adversarially Robust Models may not Transfer Better: Sufficient
  Conditions for Domain Transferability from the View of Regularization
  Machine learning (ML) robustness and domain generalization are fundamentally
correlated: they essentially concern data distribution shifts under adversarial
and natural settings, respectively. On one hand, recent studies show that more
robust (adversarially trained) models are more generalizable. On the other
hand, there is a lack of theoretical understanding of their fundamental
connections. In this paper, we explore the relationship between regularization
and domain transferability considering different factors such as norm
regularization and data augmentations (DA). We propose a general theoretical
framework proving that factors involving the model function class
regularization are sufficient conditions for relative domain transferability.
Our analysis implies that "robustness" is neither necessary nor sufficient for
transferability; rather, robustness induced by adversarial training is a
by-product of such function class regularization. We then discuss popular DA
protocols and show when they can be viewed as the function class regularization
under certain conditions and therefore improve generalization. We conduct
extensive experiments to verify our theoretical findings and show several
counterexamples where robustness and generalization are negatively correlated
on different datasets.
