Distributed Deep Reinforcement Learning for Adaptive Medium Access and
  Modulation in Shared Spectrum
  Spectrum scarcity has led to growth in the use of unlicensed spectrum for
cellular systems. This motivates intelligent adaptive approaches to spectrum
access for both WiFi and 5G that improve upon traditional carrier sensing and
listen-before-talk methods. We study decentralized contention-based medium
access for base stations (BSs) of a single Radio Access Technology (RAT)
operating on unlicensed shared spectrum. We devise a learning-based algorithm
for both contention and adaptive modulation that attempts to maximize a
network-wide downlink throughput objective. We formulate and develop novel
distributed implementations of two deep reinforcement learning approaches -
Deep Q Networks and Proximal Policy Optimization - modelled on a two stage
Markov decision process. Empirically, we find the (proportional fairness)
reward accumulated by the policy gradient approach to be significantly higher
than even a genie-aided adaptive energy detection threshold. Our approaches are
further validated by improved sum and peak throughput. The scalability of our
approach to large networks is demonstrated via an improved cumulative reward
earned on both indoor and outdoor layouts with a large number of BSs.
