Multiplayer Multi-armed Bandits for Optimal Assignment in Heterogeneous
  Networks
  We consider an ad hoc network where multiple users access the same set of
channels. The channel characteristics are unknown and could be different for
each user (heterogeneous). No controller is available to coordinate channel
selections by the users, and if multiple users select the same channel, they
collide and none of them receive any rate (or reward). For such a completely
decentralized network we develop algorithms that aim to achieve optimal network
throughput. Due to lack of any direct communication between the users, we allow
each user to exchange information by transmitting in a specific pattern and
sense such transmissions from others. However, such transmissions and sensing
for information exchange do not add to network throughput. For the wideband
sensing and narrowband sensing scenarios, we first develop explore-and-commit
algorithms that converge to near-optimal allocation with high probability in a
small number of rounds. Building on this, we develop an algorithm that gives
logarithmic regret, even when the number of users changes with time. We
validate our claims through extensive experiments and show that our algorithms
perform significantly better than the state-of-the-art CSM-MAB, dE3 and dE3-TS
algorithms.
