A Reinforcement Learning Approach for Transient Control of Liquid Rocket
  Engines
  Nowadays, liquid rocket engines use closed-loop control at most near steady
operating conditions. The control of the transient phases is traditionally
performed in open-loop due to highly nonlinear system dynamics. This situation
is unsatisfactory, in particular for reusable engines. The open-loop control
system cannot provide optimal engine performance due to external disturbances
or the degeneration of engine components over time. In this paper, we study a
deep reinforcement learning approach for optimal control of a generic
gas-generator engine's continuous start-up phase. It is shown that the learned
policy can reach different steady-state operating points and convincingly adapt
to changing system parameters. A quantitative comparison with carefully tuned
open-loop sequences and PID controllers is included. The deep reinforcement
learning controller achieves the highest performance and requires only minimal
computational effort to calculate the control action, which is a big advantage
over approaches that require online optimization, such as model predictive
control. control.
