Convolutional Neural Networks For Automatic State-Time Feature
  Extraction in Reinforcement Learning Applied to Residential Load Control
  Direct load control of a heterogeneous cluster of residential demand
flexibility sources is a high-dimensional control problem with partial
observability. This work proposes a novel approach that uses a convolutional
neural network to extract hidden state-time features to mitigate the curse of
partial observability. More specific, a convolutional neural network is used as
a function approximator to estimate the state-action value function or
Q-function in the supervised learning step of fitted Q-iteration. The approach
is evaluated in a qualitative simulation, comprising a cluster of
thermostatically controlled loads that only share their air temperature, whilst
their envelope temperature remains hidden. The simulation results show that the
presented approach is able to capture the underlying hidden features and
successfully reduce the electricity cost the cluster.
