Autonomous Maintenance in IoT Networks via AoI-driven Deep Reinforcement
  Learning
  Internet of Things (IoT) with its growing number of deployed devices and
applications raises significant challenges for network maintenance procedures.
In this work, we formulate a problem of autonomous maintenance in IoT networks
as a Partially Observable Markov Decision Process. Subsequently, we utilize
Deep Reinforcement Learning algorithms (DRL) to train agents that decide if a
maintenance procedure is in order or not and, in the former case, the proper
type of maintenance needed. To avoid wasting the scarce resources of IoT
networks we utilize the Age of Information (AoI) metric as a reward signal for
the training of the smart agents. AoI captures the freshness of the sensory
data which are transmitted by the IoT sensors as part of their normal service
provision. Numerical results indicate that AoI integrates enough information
about the past and present states of the system to be successfully used in the
training of smart agents for the autonomous maintenance of the network.
