Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration
  for Mean-Field Reinforcement Learning
  Multi-agent reinforcement learning (MARL) achieves significant empirical
successes. However, MARL suffers from the curse of many agents. In this paper,
we exploit the symmetry of agents in MARL. In the most generic form, we study a
mean-field MARL problem. Such a mean-field MARL is defined on mean-field
states, which are distributions that are supported on continuous space. Based
on the mean embedding of the distributions, we propose MF-FQI algorithm that
solves the mean-field MARL and establishes a non-asymptotic analysis for MF-FQI
algorithm. We highlight that MF-FQI algorithm enjoys a "blessing of many
agents" property in the sense that a larger number of observed agents improves
the performance of MF-FQI algorithm.
