Scalable Deep Reinforcement Learning for Routing and Spectrum Access in
  Physical Layer
  This paper proposes a novel scalable reinforcement learning approach for
simultaneous routing and spectrum access in wireless ad-hoc networks. In most
previous works on reinforcement learning for network optimization, the network
topology is assumed to be fixed, and a different agent is trained for each
transmission node -- this limits scalability and generalizability. Further,
routing and spectrum access are typically treated as separate tasks. Moreover,
the optimization objective is usually a cumulative metric along the route,
e.g., number of hops or delay. In this paper, we account for the physical-layer
signal-to-interference-plus-noise ratio (SINR) in a wireless network and
further show that bottleneck objective such as the minimum SINR along the route
can also be optimized effectively using reinforcement learning. Specifically,
we propose a scalable approach in which a single agent is associated with each
flow and makes routing and spectrum access decisions as it moves along the
frontier nodes. The agent is trained according to the physical-layer
characteristics of the environment using a novel rewarding scheme based on the
Monte Carlo estimation of the future bottleneck SINR. It learns to avoid
interference by intelligently making joint routing and spectrum allocation
decisions based on the geographical location information of the neighbouring
nodes.
