Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy
  Management under Communication Failures
  Microgrids (MGs) are important players for the future transactive energy
systems where a number of intelligent Internet of Things (IoT) devices interact
for energy management in the smart grid. Although there have been many works on
MG energy management, most studies assume a perfect communication environment,
where communication failures are not considered. In this paper, we consider the
MG as a multi-agent environment with IoT devices in which AI agents exchange
information with their peers for collaboration. However, the collaboration
information may be lost due to communication failures or packet loss. Such
events may affect the operation of the whole MG. To this end, we propose a
multi-agent Bayesian deep reinforcement learning (BA-DRL) method for MG energy
management under communication failures. We first define a multi-agent
partially observable Markov decision process (MA-POMDP) to describe agents
under communication failures, in which each agent can update its beliefs on the
actions of its peers. Then, we apply a double deep Q-learning (DDQN)
architecture for Q-value estimation in BA-DRL, and propose a belief-based
correlated equilibrium for the joint-action selection of multi-agent BA-DRL.
Finally, the simulation results show that BA-DRL is robust to both power supply
uncertainty and communication failure uncertainty. BA-DRL has 4.1% and 10.3%
higher reward than Nash Deep Q-learning (Nash-DQN) and alternating direction
method of multipliers (ADMM) respectively under 1% communication failure
probability.
