Distributed Proximal Policy Optimization for Contention-Based Spectrum
  Access
  The increasing number of wireless devices operating in unlicensed spectrum
motivates the development of intelligent adaptive approaches to spectrum access
that go beyond traditional carrier sensing. We develop a novel distributed
implementation of a policy gradient method known as Proximal Policy
Optimization modelled on a two stage Markov decision process that enables such
an intelligent approach, and still achieves decentralized contention-based
medium access. In each time slot, a base station (BS) uses information from
spectrum sensing and reception quality to autonomously decide whether or not to
transmit on a given resource, with the goal of maximizing proportional fairness
network-wide. Empirically, we find the proportional fairness reward accumulated
by the policy gradient approach to be significantly higher than even a
genie-aided adaptive energy detection threshold. This is further validated by
the improved sum and maximum user throughputs achieved by our approach.
