Constrained Model-Free Reinforcement Learning for Process Optimization
  Reinforcement learning (RL) is a control approach that can handle nonlinear
stochastic optimal control problems. However, despite the promise exhibited, RL
has yet to see marked translation to industrial practice primarily due to its
inability to satisfy state constraints. In this work we aim to address this
challenge. We propose an 'oracle'-assisted constrained Q-learning algorithm
that guarantees the satisfaction of joint chance constraints with a high
probability, which is crucial for safety critical tasks. To achieve this,
constraint tightening (backoffs) are introduced and adjusted using Broyden's
method, hence making them self-tuned. This results in a general methodology
that can be imbued into approximate dynamic programming-based algorithms to
ensure constraint satisfaction with high probability. Finally, we present case
studies that analyze the performance of the proposed approach and compare this
algorithm with model predictive control (MPC). The favorable performance of
this algorithm signifies a step toward the incorporation of RL into real world
optimization and control of engineering systems, where constraints are
essential in ensuring safety.
