DeepThermal: Combustion Optimization for Thermal Power Generating Units
  Using Offline Reinforcement Learning
  Optimizing the combustion efficiency of a thermal power generating unit
(TPGU) is a highly challenging and critical task in the energy industry. We
develop a new data-driven AI system, namely DeepThermal, to optimize the
combustion control strategy for TPGUs. At its core, is a new model-based
offline reinforcement learning (RL) framework, called MORE, which leverages
historical operational data of a TGPU to solve a highly complex constrained
Markov decision process problem via purely offline training. In DeepThermal, we
first learn a data-driven combustion process simulator from the offline
dataset. The RL agent of MORE is then trained by combining real historical data
as well as carefully filtered and processed simulation data through a novel
restrictive exploration scheme. DeepThermal has been successfully deployed in
four large coal-fired thermal power plants in China. Real-world experiments
show that DeepThermal effectively improves the combustion efficiency of TPGUs.
We also report the superior performance of MORE by comparing with the
state-of-the-art algorithms on the standard offline RL benchmarks.
