Learning Efficient Multi-Agent Cooperative Visual Exploration
  We tackle the problem of cooperative visual exploration where multiple agents
need to jointly explore unseen regions as fast as possible based on visual
signals. Classical planning-based methods often suffer from expensive
computation overhead at each step and a limited expressiveness of complex
cooperation strategy. By contrast, reinforcement learning (RL) has recently
become a popular paradigm for tackling this challenge due to its modeling
capability of arbitrarily complex strategies and minimal inference overhead. In
this paper, we extend the state-of-the-art single-agent visual navigation
method, Active Neural SLAM (ANS), to the multi-agent setting by introducing a
novel RL-based planning module, Multi-agent Spatial Planner (MSP).MSP leverages
a transformer-based architecture, Spatial-TeamFormer, which effectively
captures spatial relations and intra-agent interactions via hierarchical
spatial self-attentions. In addition, we also implement a few multi-agent
enhancements to process local information from each agent for an aligned
spatial representation and more precise planning. Finally, we perform policy
distillation to extract a meta policy to significantly improve the
generalization capability of final policy. We call this overall solution,
Multi-Agent Active Neural SLAM (MAANS). MAANS substantially outperforms
classical planning-based baselines for the first time in a photo-realistic 3D
simulator, Habitat. Code and videos can be found at
https://sites.google.com/view/maans.
