A Deep Reinforcement Learning Framework for Contention-Based Spectrum
  Sharing
  The increasing number of wireless devices operating in unlicensed spectrum
motivates the development of intelligent adaptive approaches to spectrum
access. We consider decentralized contention-based medium access for base
stations (BSs) operating on unlicensed shared spectrum, where each BS
autonomously decides whether or not to transmit on a given resource. The
contention decision attempts to maximize not its own downlink throughput, but
rather a network-wide objective. We formulate this problem as a decentralized
partially observable Markov decision process with a novel reward structure that
provides long term proportional fairness in terms of throughput. We then
introduce a two-stage Markov decision process in each time slot that uses
information from spectrum sensing and reception quality to make a medium access
decision. Finally, we incorporate these features into a distributed
reinforcement learning framework for contention-based spectrum access. Our
formulation provides decentralized inference, online adaptability and also
caters to partial observability of the environment through recurrent
Q-learning. Empirically, we find its maximization of the proportional fairness
metric to be competitive with a genie-aided adaptive energy detection
threshold, while being robust to channel fading and small contention windows.
