Perception Prioritized Training of Diffusion Models
  Diffusion models learn to restore noisy data, which is corrupted with
different levels of noise, by optimizing the weighted sum of the corresponding
loss terms, i.e., denoising score matching loss. In this paper, we show that
restoring data corrupted with certain noise levels offers a proper pretext task
for the model to learn rich visual concepts. We propose to prioritize such
noise levels over other levels during training, by redesigning the weighting
scheme of the objective function. We show that our simple redesign of the
weighting scheme significantly improves the performance of diffusion models
regardless of the datasets, architectures, and sampling strategies.
