Multimodal Image-to-Image Translation via Mutual Information Estimation
  and Maximization
  Multimodal image-to-image translation (I2IT) aims to learn a conditional
distribution that explores multiple possible images in the target domain given
an input image in the source domain. Conditional generative adversarial
networks (cGANs) are often adopted for modeling such a conditional
distribution. However, cGANs are prone to ignore the latent code and learn a
unimodal distribution in conditional image synthesis, which is also known as
the mode collapse issue of GANs. To solve the problem, we propose a simple yet
effective method that explicitly estimates and maximizes the mutual information
between the latent code and the output image in cGANs by using a deep mutual
information neural estimator in this paper. Maximizing the mutual information
strengthens the statistical dependency between the latent code and the output
image, which prevents the generator from ignoring the latent code and
encourages cGANs to fully utilize the latent code for synthesizing diverse
results. Our method not only provides a new perspective from information theory
to improve diversity for I2IT but also achieves disentanglement between the
source domain content and the target domain style for free.
