Online Adaptation to Label Distribution Shift
  Machine learning models often encounter distribution shifts when deployed in
the real world. In this paper, we focus on adaptation to label distribution
shift in the online setting, where the test-time label distribution is
continually changing and the model must dynamically adapt to it without
observing the true label. Leveraging a novel analysis, we show that the lack of
true label does not hinder estimation of the expected test loss, which enables
the reduction of online label shift adaptation to conventional online learning.
Informed by this observation, we propose adaptation algorithms inspired by
classical online learning techniques such as Follow The Leader (FTL) and Online
Gradient Descent (OGD) and derive their regret bounds. We empirically verify
our findings under both simulated and real world label distribution shifts and
show that OGD is particularly effective and robust to a variety of challenging
label shift scenarios.
