Direct Load Control of Thermostatically Controlled Loads Based on Sparse
  Observations Using Deep Reinforcement Learning
  This paper considers a demand response agent that must find a near-optimal
sequence of decisions based on sparse observations of its environment.
Extracting a relevant set of features from these observations is a challenging
task and may require substantial domain knowledge. One way to tackle this
problem is to store sequences of past observations and actions in the state
vector, making it high dimensional, and apply techniques from deep learning.
This paper investigates the capabilities of different deep learning techniques,
such as convolutional neural networks and recurrent neural networks, to extract
relevant features for finding near-optimal policies for a residential heating
system and electric water heater that are hindered by sparse observations. Our
simulation results indicate that in this specific scenario, feeding sequences
of time-series to an LSTM network, which is a specific type of recurrent neural
network, achieved a higher performance than stacking these time-series in the
input of a convolutional neural network or deep neural network.
