GAN Cocktail: mixing GANs without dataset access
  Today's generative models are capable of synthesizing high-fidelity images,
but each model specializes on a specific target domain. This raises the need
for model merging: combining two or more pretrained generative models into a
single unified one. In this work we tackle the problem of model merging, given
two constraints that often come up in the real world: (1) no access to the
original training data, and (2) without increasing the size of the neural
network. To the best of our knowledge, model merging under these constraints
has not been studied thus far. We propose a novel, two-stage solution. In the
first stage, we transform the weights of all the models to the same parameter
space by a technique we term model rooting. In the second stage, we merge the
rooted models by averaging their weights and fine-tuning them for each specific
domain, using only data generated by the original trained models. We
demonstrate that our approach is superior to baseline methods and to existing
transfer learning techniques, and investigate several applications.
