Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning
  In this paper, we derive a temporal arbitrage policy for storage via
reinforcement learning. Real-time price arbitrage is an important source of
revenue for storage units, but designing good strategies have proven to be
difficult because of the highly uncertain nature of the prices. Instead of
current model predictive or dynamic programming approaches, we use
reinforcement learning to design an optimal arbitrage policy. This policy is
learned through repeated charge and discharge actions performed by the storage
unit through updating a value matrix. We design a reward function that does not
only reflect the instant profit of charge/discharge decisions but also
incorporate the history information. Simulation results demonstrate that our
designed reward function leads to significant performance improvement compared
with existing algorithms.
