Data-Driven Learning and Load Ensemble Control
  Demand response (DR) programs aim to engage distributed small-scale flexible
loads, such as thermostatically controllable loads (TCLs), to provide various
grid support services. Linearly Solvable Markov Decision Process (LS-MDP), a
variant of the traditional MDP, is used to model aggregated TCLs. Then, a
model-free reinforcement learning technique called Z-learning is applied to
learn the value function and derive the optimal policy for the DR aggregator to
control TCLs. The learning process is robust against uncertainty that arises
from estimating the passive dynamics of the aggregated TCLs. The efficiency of
this data-driven learning is demonstrated through simulations on Heating,
Cooling & Ventilation (HVAC) units in a testbed neighborhood of residential
houses.
