Optimal Tap Setting of Voltage Regulation Transformers Using Batch
  Reinforcement Learning
  In this paper, we address the problem of setting the tap positions of load
tap changers (LTCs) for voltage regulation in radial power distribution systems
under uncertain load dynamics. The objective is to find a policy to determine
the tap positions that only uses measurements of voltage magnitudes and
topology information so as to minimize the voltage deviation across the system.
We formulate this problem as a Markov decision process (MDP), and propose a
batch reinforcement learning (RL) algorithm to solve it. By taking advantage of
a linearized power flow model, we propose an effective algorithm to estimate
the voltage magnitudes under different tap settings, which allows the RL
algorithm to explore the state and action spaces freely offline without
impacting the system operation. To circumvent the "curse of dimensionality"
resulted from the large state and action spaces, we propose a sequential
learning algorithm to learn an action-value function for each LTC, based on
which the optimal tap positions can be directly determined. The effectiveness
of the proposed algorithm is validated via numerical simulations on the IEEE
13-bus and 123-bus distribution test feeders.
