A storage expansion planning framework using reinforcement learning and
  simulation-based optimization
  In the wake of the highly electrified future ahead of us, the role of energy
storage is crucial wherever distributed generation is abundant, such as in
microgrid settings. Given the variety of storage options that are becoming more
and more economical, determining which type of storage technology to invest in,
along with the appropriate timing and capacity becomes a critical research
question. It is inevitable that these problems will continue to become
increasingly relevant in the future and require strategic planning and holistic
and modern frameworks in order to be solved. Reinforcement Learning algorithms
have already proven to be successful in problems where sequential
decision-making is inherent. In the operations planning area, these algorithms
are already used but mostly in short-term problems with well-defined
constraints. On the contrary, we expand and tailor these techniques to
long-term planning by utilizing model-free algorithms combined with
simulation-based models. A model and expansion plan have been developed to
optimally determine microgrid designs as they evolve to dynamically react to
changing conditions and to exploit energy storage capabilities. We show that it
is possible to derive better engineering solutions that would point to the
types of energy storage units which could be at the core of future microgrid
applications. Another key finding is that the optimal storage capacity
threshold for a system depends heavily on the price movements of the available
storage units. By utilizing the proposed approaches, it is possible to model
inherent problem uncertainties and optimize the whole streamline of sequential
investment decision-making.
