Scalable Voltage Control using Structure-Driven Hierarchical Deep
  Reinforcement Learning
  This paper presents a novel hierarchical deep reinforcement learning (DRL)
based design for the voltage control of power grids. DRL agents are trained for
fast, and adaptive selection of control actions such that the voltage recovery
criterion can be met following disturbances. Existing voltage control
techniques suffer from the issues of speed of operation, optimal coordination
between different locations, and scalability. We exploit the area-wise division
structure of the power system to propose a hierarchical DRL design that can be
scaled to the larger grid models. We employ an enhanced augmented random search
algorithm that is tailored for the voltage control problem in a two-level
architecture. We train area-wise decentralized RL agents to compute lower-level
policies for the individual areas, and concurrently train a higher-level DRL
agent that uses the updates of the lower-level policies to efficiently
coordinate the control actions taken by the lower-level agents. Numerical
experiments on the IEEE benchmark 39-bus model with 3 areas demonstrate the
advantages and various intricacies of the proposed hierarchical approach.
