Optimal PID and Antiwindup Control Design as a Reinforcement Learning
  Problem
  Deep reinforcement learning (DRL) has seen several successful applications to
process control. Common methods rely on a deep neural network structure to
model the controller or process. With increasingly complicated control
structures, the closed-loop stability of such methods becomes less clear. In
this work, we focus on the interpretability of DRL control methods. In
particular, we view linear fixed-structure controllers as shallow neural
networks embedded in the actor-critic framework. PID controllers guide our
development due to their simplicity and acceptance in industrial practice. We
then consider input saturation, leading to a simple nonlinear control
structure. In order to effectively operate within the actuator limits we then
incorporate a tuning parameter for anti-windup compensation. Finally, the
simplicity of the controller allows for straightforward initialization. This
makes our method inherently stabilizing, both during and after training, and
amenable to known operational PID gains.
