Weighted Double Deep Multiagent Reinforcement Learning in Stochastic
  Cooperative Environments
  Recently, multiagent deep reinforcement learning (DRL) has received
increasingly wide attention. Existing multiagent DRL algorithms are inefficient
when facing with the non-stationarity due to agents update their policies
simultaneously in stochastic cooperative environments. This paper extends the
recently proposed weighted double estimator to the multiagent domain and
propose a multiagent DRL framework, named weighted double deep Q-network
(WDDQN). By utilizing the weighted double estimator and the deep neural
network, WDDQN can not only reduce the bias effectively but also be extended to
scenarios with raw visual inputs. To achieve efficient cooperation in the
multiagent domain, we introduce the lenient reward network and the scheduled
replay strategy. Experiments show that the WDDQN outperforms the existing DRL
and multiaent DRL algorithms, i.e., double DQN and lenient Q-learning, in terms
of the average reward and the convergence rate in stochastic cooperative
environments.
