Decentralized Multi-Agent Actor-Critic with Generative Inference
  Recent multi-agent actor-critic methods have utilized centralized training
with decentralized execution to address the non-stationarity of co-adapting
agents. This training paradigm constrains learning to the centralized phase
such that only pre-learned policies may be used during the decentralized phase,
which performs poorly when agent communications are delayed, noisy, or
disrupted. In this work, we propose a new system that can gracefully handle
partially-observable information due to communication disruptions during
decentralized execution. Our approach augments the multi-agent actor-critic
method's centralized training phase with generative modeling so that agents may
infer other agents' observations when provided with locally available context.
Our method is evaluated on three tasks that require agents to combine local and
remote observations communicated by other agents. We evaluate our approach by
introducing both partial observability during decentralized execution, and show
that decentralized training on inferred observations performs as well or better
than existing actor-critic methods.
