A Neural Tangent Kernel Perspective of GANs
  We propose a novel theoretical framework of analysis for Generative
Adversarial Networks (GANs). We start by pointing out a fundamental flaw in
previous theoretical analyses that leads to ill-defined gradients for the
discriminator. We overcome this issue which impedes a principled study of GAN
training, solving it within our framework by taking into account the
discriminator's architecture. To this end, we leverage the theory of
infinite-width neural networks for the discriminator via its Neural Tangent
Kernel. We characterize the trained discriminator for a wide range of losses
and establish general differentiability properties of the network. Moreover, we
derive new insights about the generated distribution's flow during training,
advancing our understanding of GAN dynamics. We empirically corroborate these
results via a publicly released analysis toolkit based on our framework,
unveiling intuitions that are consistent with current GAN practice.
