Online Learning of Rested and Restless Bandits
  In this paper we study the online learning problem involving rested and
restless multiarmed bandits with multiple plays. The system consists of a
single player/user and a set of K finite-state discrete-time Markov chains
(arms) with unknown state spaces and statistics. At each time step the player
can play M arms. The objective of the user is to decide for each step which M
of the K arms to play over a sequence of trials so as to maximize its long term
reward. The restless multiarmed bandit is particularly relevant to the
application of opportunistic spectrum access (OSA), where a (secondary) user
has access to a set of K channels, each of time-varying condition as a result
of random fading and/or certain primary users' activities.
