PriorityCut: Occlusion-guided Regularization for Warp-based Image
  Animation
  Image animation generates a video of a source image following the motion of a
driving video. State-of-the-art self-supervised image animation approaches warp
the source image according to the motion of the driving video and recover the
warping artifacts by inpainting. These approaches mostly use vanilla
convolution for inpainting, and vanilla convolution does not distinguish
between valid and invalid pixels. As a result, visual artifacts are still
noticeable after inpainting. CutMix is a state-of-the-art regularization
strategy that cuts and mixes patches of images and is widely studied in
different computer vision tasks. Among the remaining computer vision tasks,
warp-based image animation is one of the fields that the effects of CutMix have
yet to be studied. This paper first presents a preliminary study on the effects
of CutMix on warp-based image animation. We observed in our study that CutMix
helps improve only pixel values, but disturbs the spatial relationships between
pixels. Based on such observation, we propose PriorityCut, a novel augmentation
approach that uses the top-k percent occluded pixels of the foreground to
regularize warp-based image animation. By leveraging the domain knowledge in
warp-based image animation, PriorityCut significantly reduces the warping
artifacts in state-of-the-art warp-based image animation models on diverse
datasets.
