Deep Reinforcement Learning-Based Beam Tracking for Low-Latency Services
  in Vehicular Networks
  Ultra-Reliable and Low-Latency Communications (URLLC) services in vehicular
networks on millimeter-wave bands present a significant challenge, considering
the necessity of constantly adjusting the beam directions. Conventional methods
are mostly based on classical control theory, e.g., Kalman filter and its
variations, which mainly deal with stationary scenarios. Therefore, severe
application limitations exist, especially with complicated, dynamic
Vehicle-to-Everything (V2X) channels. This paper gives a thorough study of this
subject, by first modifying the classical approaches, e.g., Extended Kalman
Filter (EKF) and Particle Filter (PF), for non-stationary scenarios, and then
proposing a Reinforcement Learning (RL)-based approach that can achieve the
URLLC requirements in a typical intersection scenario. Simulation results based
on a commercial ray-tracing simulator show that enhanced EKF and PF methods
achieve packet delay more than $10$ ms, whereas the proposed deep RL-based
method can reduce the latency to about $6$ ms, by extracting context
information from the training data.
