Improving Augmentation and Evaluation Schemes for Semantic Image
  Synthesis
  Despite data augmentation being a de facto technique for boosting the
performance of deep neural networks, little attention has been paid to
developing augmentation strategies for generative adversarial networks (GANs).
To this end, we introduce a novel augmentation scheme designed specifically for
GAN-based semantic image synthesis models. We propose to randomly warp object
shapes in the semantic label maps used as an input to the generator. The local
shape discrepancies between the warped and non-warped label maps and images
enable the GAN to learn better the structural and geometric details of the
scene and thus to improve the quality of generated images. While benchmarking
the augmented GAN models against their vanilla counterparts, we discover that
the quantification metrics reported in the previous semantic image synthesis
studies are strongly biased towards specific semantic classes as they are
derived via an external pre-trained segmentation network. We therefore propose
to improve the established semantic image synthesis evaluation scheme by
analyzing separately the performance of generated images on the biased and
unbiased classes for the given segmentation network. Finally, we show strong
quantitative and qualitative improvements obtained with our augmentation
scheme, on both class splits, using state-of-the-art semantic image synthesis
models across three different datasets. On average across COCO-Stuff, ADE20K
and Cityscapes datasets, the augmented models outperform their vanilla
counterparts by ~3 mIoU and ~10 FID points.
