Controllable Image Synthesis via SegVAE
  Flexible user controls are desirable for content creation and image editing.
A semantic map is commonly used intermediate representation for conditional
image generation. Compared to the operation on raw RGB pixels, the semantic map
enables simpler user modification. In this work, we specifically target at
generating semantic maps given a label-set consisting of desired categories.
The proposed framework, SegVAE, synthesizes semantic maps in an iterative
manner using conditional variational autoencoder. Quantitative and qualitative
experiments demonstrate that the proposed model can generate realistic and
diverse semantic maps. We also apply an off-the-shelf image-to-image
translation model to generate realistic RGB images to better understand the
quality of the synthesized semantic maps. Furthermore, we showcase several
real-world image-editing applications including object removal, object
insertion, and object replacement.
