GANs Can Play Lottery Tickets Too
  Deep generative adversarial networks (GANs) have gained growing popularity in
numerous scenarios, while usually suffer from high parameter complexities for
resource-constrained real-world applications. However, the compression of GANs
has less been explored. A few works show that heuristically applying
compression techniques normally leads to unsatisfactory results, due to the
notorious training instability of GANs. In parallel, the lottery ticket
hypothesis shows prevailing success on discriminative models, in locating
sparse matching subnetworks capable of training in isolation to full model
performance. In this work, we for the first time study the existence of such
trainable matching subnetworks in deep GANs. For a range of GANs, we certainly
find matching subnetworks at 67%-74% sparsity. We observe that with or without
pruning discriminator has a minor effect on the existence and quality of
matching subnetworks, while the initialization weights used in the
discriminator play a significant role. We then show the powerful
transferability of these subnetworks to unseen tasks. Furthermore, extensive
experimental results demonstrate that our found subnetworks substantially
outperform previous state-of-the-art GAN compression approaches in both image
generation (e.g. SNGAN) and image-to-image translation GANs (e.g. CycleGAN).
Codes available at https://github.com/VITA-Group/GAN-LTH.
