Macro-action Multi-time scale Dynamic Programming for Energy Management
  in Buildings with Phase Change Materials
  This paper focuses on energy management in buildings with phase change
material (PCM), which is primarily used to improve thermal performance, but can
also serve as an energy storage system. In this setting, optimal scheduling of
an HVAC system is challenging because of the nonlinear and non-convex
characteristics of the PCM, which makes solving the corresponding optimization
problem using conventional optimization techniques impractical. Instead, we use
dynamic programming (DP) to deal with the nonlinear nature of the PCM. To
overcome DP's curse of dimensionality, this paper proposes a novel methodology
to reduce the computational burden, while maintaining the quality of the
solution. Specifically, the method incorporates approaches from sequential
decision making in artificial intelligence, including macro actions and
multi-time scale Markov decision processes, coupled with an underlying
state-space approximation to reduce the state-space and action-space size. The
performance of the method is demonstrated on an energy management problem for a
typical residential building located in Sydney, Australia. The results
demonstrate that the proposed method performs well with a computational
speed-up of up to 12,900 times compared to the direct application of DP.
