Stabilizing Voltage in Power Distribution Networks via Multi-Agent
  Reinforcement Learning with Transformer
  The increased integration of renewable energy poses a slew of technical
challenges for the operation of power distribution networks. Among them,
voltage fluctuations caused by the instability of renewable energy are
receiving increasing attention. Utilizing MARL algorithms to coordinate
multiple control units in the grid, which is able to handle rapid changes of
power systems, has been widely studied in active voltage control task recently.
However, existing approaches based on MARL ignore the unique nature of the grid
and achieve limited performance. In this paper, we introduce the transformer
architecture to extract representations adapting to power network problems and
propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to
stabilize voltage in power distribution networks. In addition, we adopt a novel
auxiliary-task training process tailored to the voltage control task, which
improves the sample efficiency and facilitating the representation learning of
the transformer-based model. We couple T-MAAC with different multi-agent
actor-critic algorithms, and the consistent improvements on the active voltage
control task demonstrate the effectiveness of the proposed method.
