Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA
  Reinforcement Learning
  Sensors are being extensively deployed and are expected to expand at
significant rates in the coming years. They typically generate a large volume
of data on the internet of things (IoT) application areas like smart cities,
intelligent traffic systems, smart grid, and e-health. Cloud, edge and fog
computing are potential and competitive strategies for collecting, processing,
and distributing IoT data. However, cloud, edge, and fog-based solutions need
to tackle the distribution of a high volume of IoT data efficiently through
constrained and limited resource network infrastructures. This paper addresses
the issue of conveying a massive volume of IoT data through a network with
limited communications resources (bandwidth) using a cognitive communications
resource allocation based on Reinforcement Learning (RL) with SARSA algorithm.
The proposed network infrastructure (PSIoTRL) uses a Publish/ Subscribe
architecture to access massive and highly distributed IoT data. It is
demonstrated that the PSIoTRL bandwidth allocation for buffer flushing based on
SARSA enhances the IoT aggregator buffer occupation and network link
utilization. The PSIoTRL dynamically adapts the IoT aggregator traffic flushing
according to the Pub/Sub topic's priority and network constraint requirements.
