Parameter Adaptation for Joint Distribution Shifts
While different methods exist to tackle distinct types of distribution shift,
such as label shift (in the form of adversarial attacks) or domain shift,
tackling the joint shift setting is still an open problem. Through the study of
a joint distribution shift manifesting both adversarial and domain-specific
perturbations, we not only show that a joint shift worsens model performance
compared to their individual shifts, but that the use of a similar domain
worsens performance than a dissimilar domain. To curb the performance drop, we
study the use of perturbation sets motivated by input and parameter space
bounds, and adopt a meta learning strategy (hypernetworks) to model parameters
w.r.t. test-time inputs to recover performance.