BGC: Multi-Agent Group Belief with Graph Clustering
  Recent advances have witnessed that value decomposed-based multi-agent
reinforcement learning methods make an efficient performance in coordination
tasks. Most current methods assume that agents can make communication to assist
decisions, which is impractical in some situations. In this paper, we propose a
semi-communication method to enable agents can exchange information without
communication. Specifically, we introduce a group concept to help agents
learning a belief which is a type of consensus. With this consensus, adjacent
agents tend to accomplish similar sub-tasks to achieve cooperation. We design a
novel agent structure named Belief in Graph Clustering(BGC), composed of an
agent characteristic module, a belief module, and a fusion module. To represent
each agent characteristic, we use an MLP-based characteristic module to
generate agent unique features. Inspired by the neighborhood cognitive
consistency, we propose a group-based module to divide adjacent agents into a
small group and minimize in-group agents' beliefs to accomplish similar
sub-tasks. Finally, we use a hyper-network to merge these features and produce
agent actions. To overcome the agent consistent problem brought by GAT, a split
loss is introduced to distinguish different agents. Results reveal that the
proposed method achieves a significant improvement in the SMAC benchmark.
Because of the group concept, our approach maintains excellent performance with
an increase in the number of agents.
