Reinforcement Learning Based Vehicle-cell Association Algorithm for
  Highly Mobile Millimeter Wave Communication
  Vehicle-to-everything (V2X) communication is a growing area of communication
with a variety of use cases. This paper investigates the problem of
vehicle-cell association in millimeter wave (mmWave) communication networks.
The aim is to maximize the time average rate per vehicular user (VUE) while
ensuring a target minimum rate for all VUEs with low signaling overhead. We
first formulate the user (vehicle) association problem as a discrete non-convex
optimization problem. Then, by leveraging tools from machine learning,
specifically distributed deep reinforcement learning (DDRL) and the
asynchronous actor critic algorithm (A3C), we propose a low complexity
algorithm that approximates the solution of the proposed optimization problem.
The proposed DDRL-based algorithm endows every road side unit (RSU) with a
local RL agent that selects a local action based on the observed input state.
Actions of different RSUs are forwarded to a central entity, that computes a
global reward which is then fed back to RSUs. It is shown that each
independently trained RL performs the vehicle-RSU association action with low
control overhead and less computational complexity compared to running an
online complex algorithm to solve the non-convex optimization problem. Finally,
simulation results show that the proposed solution achieves up to 15\% gains in
terms of sum rate and 20\% reduction in VUE outages compared to several
baseline designs.
