A Systematic Survey of Regularization and Normalization in GANs
  Generative Adversarial Networks (GANs) have been widely applied in different
scenarios thanks to the development of deep neural networks. The original GAN
was proposed based on the non-parametric assumption of the infinite capacity of
networks. However, it is still unknown whether GANs can generate realistic
samples without any prior information. Due to the overconfident assumption,
many issues remain unaddressed in GANs' training, such as non-convergence, mode
collapses, gradient vanishing. Regularization and normalization are common
methods of introducing prior information to stabilize training and improve
discrimination. Although a handful number of regularization and normalization
methods have been proposed for GANs, to the best of our knowledge, there exists
no comprehensive survey which primarily focuses on objectives and development
of these methods, apart from some in-comprehensive and limited scope studies.
In this work, we conduct a comprehensive survey on the regularization and
normalization techniques from different perspectives of GANs training. First,
we systematically describe different perspectives of GANs training and thus
obtain the different objectives of regularization and normalization. Based on
these objectives, we propose a new taxonomy. Furthermore, we compare the
performance of the mainstream methods on different datasets and investigate the
regularization and normalization techniques that have been frequently employed
in SOTA GANs. Finally, we highlight potential future directions of research in
this domain.
