Improving Text-to-Image Synthesis Using Contrastive Learning
  The goal of text-to-image synthesis is to generate a visually realistic image
that matches a given text description. In practice, the captions annotated by
humans for the same image have large variance in terms of contents and the
choice of words. The linguistic discrepancy between the captions of the
identical image leads to the synthetic images deviating from the ground truth.
To address this issue, we propose a contrastive learning approach to improve
the quality and enhance the semantic consistency of synthetic images. In the
pretraining stage, we utilize the contrastive learning approach to learn the
consistent textual representations for the captions corresponding to the same
image. Furthermore, in the following stage of GAN training, we employ the
contrastive learning method to enhance the consistency between the generated
images from the captions related to the same image. We evaluate our approach
over two popular text-to-image synthesis models, AttnGAN and DM-GAN, on
datasets CUB and COCO, respectively. Experimental results have shown that our
approach can effectively improve the quality of synthetic images in terms of
three metrics: IS, FID and R-precision. Especially, on the challenging COCO
dataset, our approach boosts the FID signifcantly by 29.60% over AttnGAN and by
21.96% over DM-GAN.
