A Reinforcement Learning Approach for the Continuous Electricity Market
  of Germany: Trading from the Perspective of a Wind Park Operator
  With the rising extension of renewable energies, the intraday electricity
markets have recorded a growing popularity amongst traders as well as electric
utilities to cope with the induced volatility of the energy supply. Through
their short trading horizon and continuous nature, the intraday markets offer
the ability to adjust trading decisions from the day-ahead market or reduce
trading risk in a short-term notice. Producers of renewable energies utilize
the intraday market to lower their forecast risk, by modifying their provided
capacities based on current forecasts. However, the market dynamics are complex
due to the fact that the power grids have to remain stable and electricity is
only partly storable. Consequently, robust and intelligent trading strategies
are required that are capable to operate in the intraday market. In this work,
we propose a novel autonomous trading approach based on Deep Reinforcement
Learning (DRL) algorithms as a possible solution. For this purpose, we model
the intraday trade as a Markov Decision Problem (MDP) and employ the Proximal
Policy Optimization (PPO) algorithm as our DRL approach. A simulation framework
is introduced that enables the trading of the continuous intraday price in a
resolution of one minute steps. We test our framework in a case study from the
perspective of a wind park operator. We include next to general trade
information both price and wind forecasts. On a test scenario of German
intraday trading results from 2018, we are able to outperform multiple
baselines with at least 45.24% improvement, showing the advantage of the DRL
algorithm. However, we also discuss limitations and enhancements of the DRL
agent, in order to increase the performance in future works.
