Learning to Predict Layout-to-image Conditional Convolutions for
  Semantic Image Synthesis
  Semantic image synthesis aims at generating photorealistic images from
semantic layouts. Previous approaches with conditional generative adversarial
networks (GAN) show state-of-the-art performance on this task, which either
feed the semantic label maps as inputs to the generator, or use them to
modulate the activations in normalization layers via affine transformations. We
argue that convolutional kernels in the generator should be aware of the
distinct semantic labels at different locations when generating images. In
order to better exploit the semantic layout for the image generator, we propose
to predict convolutional kernels conditioned on the semantic label map to
generate the intermediate feature maps from the noise maps and eventually
generate the images. Moreover, we propose a feature pyramid semantics-embedding
discriminator, which is more effective in enhancing fine details and semantic
alignments between the generated images and the input semantic layouts than
previous multi-scale discriminators. We achieve state-of-the-art results on
both quantitative metrics and subjective evaluation on various semantic
segmentation datasets, demonstrating the effectiveness of our approach.
