SA-MATD3:Self-attention-based multi-agent continuous control method in
  cooperative environments
  Cooperative problems under continuous control have always been the focus of
multi-agent reinforcement learning. Existing algorithms suffer from the problem
of uneven learning degree with the increase of the number of agents. In this
paper, a new structure for a multi-agent actor critic is proposed, and the
self-attention mechanism is applied in the critic network and the value
decomposition method used to solve the uneven problem. The proposed algorithm
makes full use of the samples in the replay memory buffer to learn the behavior
of a class of agents. First, a new update method is proposed for policy
networks that promotes learning efficiency. Second, the utilization of samples
is improved, at the same time reflecting the ability of perspective-taking
among groups. Finally, the "deceptive signal" in training is eliminated and the
learning degree among agents is more uniform than in the existing methods.
Multiple experiments were conducted in two typical scenarios of a multi-agent
particle environment. Experimental results show that the proposed algorithm can
perform better than the state-of-the-art ones, and that it exhibits higher
learning efficiency with an increasing number of agents.
