Multi-IRS-assisted Multi-Cell Uplink MIMO Communications under Imperfect
  CSI: A Deep Reinforcement Learning Approach
  Applications of intelligent reflecting surfaces (IRSs) in wireless networks
have attracted significant attention recently. Most of the relevant literature
is focused on the single cell setting where a single IRS is deployed and
perfect channel state information (CSI) is assumed. In this work, we develop a
novel methodology for multi-IRS-assisted multi-cell networks in the uplink. We
consider the scenario in which (i) channels are dynamic and (ii) only partial
CSI is available at each base station (BS); specifically, scalar effective
channel powers from only a subset of user equipments (UE). We formulate the
sum-rate maximization problem aiming to jointly optimize the IRS reflect
beamformers, BS combiners, and UE transmit powers. In casting this as a
sequential decision making problem, we propose a multi-agent deep reinforcement
learning algorithm to solve it, where each BS acts as an independent agent in
charge of tuning the local UE transmit powers, the local IRS reflect
beamformer, and its combiners. We introduce an efficient information-sharing
scheme that requires limited information exchange among neighboring BSs to cope
with the non-stationarity caused by the coupling of actions taken by multiple
BSs. Our numerical results show that our method obtains substantial improvement
in average data rate compared to baseline approaches, e.g., fixed UE transmit
power and maximum ratio combining.
