Understanding and Stabilizing GANs' Training Dynamics with Control
  Theory
  Generative adversarial networks (GANs) are effective in generating realistic
images but the training is often unstable. There are existing efforts that
model the training dynamics of GANs in the parameter space but the analysis
cannot directly motivate practically effective stabilizing methods. To this
end, we present a conceptually novel perspective from control theory to
directly model the dynamics of GANs in the function space and provide simple
yet effective methods to stabilize GANs' training. We first analyze the
training dynamic of a prototypical Dirac GAN and adopt the widely-used
closed-loop control (CLC) to improve its stability. We then extend CLC to
stabilize the training dynamic of normal GANs, where CLC is implemented as a
squared $L2$ regularizer on the output of the discriminator. Empirical results
show that our method can effectively stabilize the training and obtain
state-of-the-art performance on data generation tasks.
