Reinforcement Learning for Optimized Beam Training in Multi-Hop
  Terahertz Communications
  Communication at terahertz (THz) frequency bands is a promising solution for
achieving extremely high data rates in next-generation wireless networks. While
the THz communication is conventionally envisioned for short-range wireless
applications due to the high atmospheric absorption at THz frequencies,
multi-hop directional transmissions can be enabled to extend the communication
range. However, to realize multi-hop THz communications, conventional beam
training schemes, such as exhaustive search or hierarchical methods with a
fixed number of training levels, can lead to a very large time overhead. To
address this challenge, in this paper, a novel hierarchical beam training
scheme with dynamic training levels is proposed to optimize the performance of
multi-hop THz links. In fact, an optimization problem is formulated to maximize
the overall spectral efficiency of the multi-hop THz link by dynamically and
jointly selecting the number of beam training levels across all the constituent
single-hop links. To solve this problem in presence of unknown channel state
information, noise, and path loss, a new reinforcement learning solution based
on the multi-armed bandit (MAB) is developed. Simulation results show the fast
convergence of the proposed scheme in presence of random channels and noise.
The results also show that the proposed scheme can yield up to 75% performance
gain, in terms of spectral efficiency, compared to the conventional
hierarchical beam training with a fixed number of training levels.
