Self-Tuning Bandits over Unknown Covariate-Shifts
  Bandits with covariates, a.k.a. contextual bandits, address situations where
optimal actions (or arms) at a given time $t$, depend on a context $x_t$, e.g.,
a new patient's medical history, a consumer's past purchases. While it is
understood that the distribution of contexts might change over time, e.g., due
to seasonalities, or deployment to new environments, the bulk of studies
concern the most adversarial such changes, resulting in regret bounds that are
often worst-case in nature.
  Covariate-shift on the other hand has been considered in classification as a
middle-ground formalism that can capture mild to relatively severe changes in
distributions. We consider nonparametric bandits under such middle-ground
scenarios, and derive new regret bounds that tightly capture a continuum of
changes in context distribution. Furthermore, we show that these rates can be
adaptively attained without knowledge of the time of shift nor the amount of
shift.
