A Reinforcement Learning Approach to Age of Information in Multi-User
  Networks with HARQ
  Scheduling the transmission of time-sensitive information from a source node
to multiple users over error-prone communication channels is studied with the
goal of minimizing the long-term average age of information (AoI) at the users.
A long-term average resource constraint is imposed on the source, which limits
the average number of transmissions. The source can transmit only to a single
user at each time slot, and after each transmission, it receives an
instantaneous ACK/NACK feedback from the intended receiver, and decides when
and to which user to transmit the next update. Assuming the channel statistics
are known, the optimal scheduling policy is studied for both the standard
automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols. Then, a
reinforcement learning(RL) approach is introduced to find a near-optimal
policy, which does not assume any a priori information on the random processes
governing the channel states. Different RL methods including average-cost
SARSAwith linear function approximation (LFA), upper confidence reinforcement
learning (UCRL2), and deep Q-network (DQN) are applied and compared through
numerical simulations
