Deep Reinforcement Learning for Simultaneous Sensing and Channel Access
  in Cognitive Networks
  We consider the problem of dynamic spectrum access (DSA) in cognitive
wireless networks, where only partial observations are available to the users
due to narrowband sensing and transmissions. The cognitive network consists of
primary users (PUs) and a secondary user (SU), which operate in a time
duplexing regime. The traffic pattern for each PU is assumed to be unknown to
the SU and is modeled as a finite-memory Markov chain. Since observations are
partial, then both channel sensing and access actions affect the throughput.
The objective is to maximize the SU's long-term throughput. To achieve this
goal, we develop a novel algorithm that learns both access and sensing policies
via deep Q-learning, dubbed Double Deep Q-network for Sensing and Access
(DDQSA). To the best of our knowledge, this is the first paper that solves both
sensing and access policies for DSA via deep Q-learning. Second, we analyze the
optimal policy theoretically to validate the performance of DDQSA. Although the
general DSA problem is P-SPACE hard, we derive the optimal policy explicitly
for a common model of a cyclic user dynamics. Our results show that DDQSA
learns a policy that implements both sensing and channel access, and
significantly outperforms existing approaches.
