When, Why, and Which Pretrained GANs Are Useful?
  The literature has proposed several methods to finetune pretrained GANs on
new datasets, which typically results in higher performance compared to
training from scratch, especially in the limited-data regime. However, despite
the apparent empirical benefits of GAN pretraining, its inner mechanisms were
not analyzed in-depth, and understanding of its role is not entirely clear.
Moreover, the essential practical details, e.g., selecting a proper pretrained
GAN checkpoint, currently do not have rigorous grounding and are typically
determined by trial and error.
  This work aims to dissect the process of GAN finetuning. First, we show that
initializing the GAN training process by a pretrained checkpoint primarily
affects the model's coverage rather than the fidelity of individual samples.
Second, we explicitly describe how pretrained generators and discriminators
contribute to the finetuning process and explain the previous evidence on the
importance of pretraining both of them. Finally, as an immediate practical
benefit of our analysis, we describe a simple recipe to choose an appropriate
GAN checkpoint that is the most suitable for finetuning to a particular target
task. Importantly, for most of the target tasks, Imagenet-pretrained GAN,
despite having poor visual quality, appears to be an excellent starting point
for finetuning, resembling the typical pretraining scenario of discriminative
computer vision models.
