Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoT
  The recent success of single-agent reinforcement learning (RL) in Internet of
things (IoT) systems motivates the study of multi-agent reinforcement learning
(MARL), which is more challenging but more useful in large-scale IoT. In this
paper, we consider a voting-based MARL problem, in which the agents vote to
make group decisions and the goal is to maximize the globally averaged returns.
To this end, we formulate the MARL problem based on the linear programming form
of the policy optimization problem and propose a distributed primal-dual
algorithm to obtain the optimal solution. We also propose a voting mechanism
through which the distributed learning achieves the same sublinear convergence
rate as centralized learning. In other words, the distributed decision making
does not slow down the process of achieving global consensus on optimality.
Lastly, we verify the convergence of our proposed algorithm with numerical
simulations and conduct case studies in practical multi-agent IoT systems.
