Joint AP Probing and Scheduling: A Contextual Bandit Approach
  We consider a set of APs with unknown data rates that cooperatively serve a
mobile client. The data rate of each link is i.i.d. sampled from a distribution
that is unknown a priori. In contrast to traditional link scheduling problems
under uncertainty, we assume that in each time step, the device can probe a
subset of links before deciding which one to use. We model this problem as a
contextual bandit problem with probing (CBwP) and present an efficient
algorithm. We further establish the regret of our algorithm for links with
Bernoulli data rates. Our CBwP model is a novel extension of the classic
contextual bandit model and can potentially be applied to a large class of
sequential decision-making problems that involve joint probing and play under
uncertainty.
