Learning disconnected manifolds: a no GANs land
  Typical architectures of Generative AdversarialNetworks make use of a
unimodal latent distribution transformed by a continuous generator.
Consequently, the modeled distribution always has connected support which is
cumbersome when learning a disconnected set of manifolds. We formalize this
problem by establishing a no free lunch theorem for the disconnected manifold
learning stating an upper bound on the precision of the targeted distribution.
This is done by building on the necessary existence of a low-quality region
where the generator continuously samples data between two disconnected modes.
Finally, we derive a rejection sampling method based on the norm of generators
Jacobian and show its efficiency on several generators including BigGAN.
