Millimeter Wave Communications with an Intelligent Reflector:
  Performance Optimization and Distributional Reinforcement Learning
  In this paper, a novel framework is proposed to optimize the downlink
multi-user communication of a millimeter wave base station, which is assisted
by a reconfigurable intelligent reflector (IR). In particular, a channel
estimation approach is developed to measure the channel state information (CSI)
in real-time. First, for a perfect CSI scenario, the precoding transmission of
the BS and the reflection coefficient of the IR are jointly optimized, via an
iterative approach, so as to maximize the sum of downlink rates towards
multiple users. Next, in the imperfect CSI scenario, a distributional
reinforcement learning (DRL) approach is proposed to learn the optimal IR
reflection and maximize the expectation of downlink capacity. In order to model
the transmission rate's probability distribution, a learning algorithm, based
on quantile regression (QR), is developed, and the proposed QR-DRL method is
proved to converge to a stable distribution of downlink transmission rate.
Simulation results show that, in the error-free CSI scenario, the proposed
approach yields over 30% and 2-fold increase in the downlink sum-rate, compared
with a fixed IR reflection scheme and direct transmission scheme, respectively.
Simulation results also show that by deploying more IR elements, the downlink
sum-rate can be significantly improved. However, as the number of IR components
increases, more time is required for channel estimation, and the slope of
increase in the IR-aided transmission rate will become smaller. Furthermore,
under limited knowledge of CSI, simulation results show that the proposed
QR-DRL method, which learns a full distribution of the downlink rate, yields a
better prediction accuracy and improves the downlink rate by 10% for online
deployments, compared with a Q-learning baseline.
