D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
  Conditional generative models of high-dimensional images have many
applications, but supervision signals from conditions to images can be
expensive to acquire. This paper describes Diffusion-Decoding models with
Contrastive representations (D2C), a paradigm for training unconditional
variational autoencoders (VAEs) for few-shot conditional image generation. D2C
uses a learned diffusion-based prior over the latent representations to improve
generation and contrastive self-supervised learning to improve representation
quality. D2C can adapt to novel generation tasks conditioned on labels or
manipulation constraints, by learning from as few as 100 labeled examples. On
conditional generation from new labels, D2C achieves superior performance over
state-of-the-art VAEs and diffusion models. On conditional image manipulation,
D2C generations are two orders of magnitude faster to produce over StyleGAN2
ones and are preferred by 50% - 60% of the human evaluators in a double-blind
study.
