Optimizing Generative Adversarial Networks for Image Super Resolution
  via Latent Space Regularization
  Natural images can be regarded as residing in a manifold that is embedded in
a higher dimensional Euclidean space. Generative Adversarial Networks (GANs)
try to learn the distribution of the real images in the manifold to generate
samples that look real. But the results of existing methods still exhibit many
unpleasant artifacts and distortions even for the cases where the desired
ground truth target images are available for supervised learning such as in
single image super resolution (SISR). We probe for ways to alleviate these
problems for supervised GANs in this paper. We explicitly apply the Lipschitz
Continuity Condition (LCC) to regularize the GAN. An encoding network that maps
the image space to a new optimal latent space is derived from the LCC, and it
is used to augment the GAN as a coupling component. The LCC is also converted
to new regularization terms in the generator loss function to enforce local
invariance. The GAN is optimized together with the encoding network in an
attempt to make the generator converge to a more ideal and disentangled mapping
that can generate samples more faithful to the target images. When the proposed
models are applied to the single image super resolution problem, the results
outperform the state of the art.
