DivSwapper: Towards Diversified Patch-based Arbitrary Style Transfer
  Gram-based and patch-based approaches are two important research lines of
style transfer. Recent diversified Gram-based methods have been able to produce
multiple and diverse stylized outputs for the same content and style images.
However, as another widespread research interest, the diversity of patch-based
methods remains challenging due to the stereotyped style swapping process based
on nearest patch matching. To resolve this dilemma, in this paper, we dive into
the crux of existing patch-based methods and propose a universal and efficient
module, termed DivSwapper, for diversified patch-based arbitrary style
transfer. The key insight is to use an essential intuition that neural patches
with higher activation values could contribute more to diversity. Our
DivSwapper is plug-and-play and can be easily integrated into existing
patch-based and Gram-based methods to generate diverse results for arbitrary
styles. We conduct theoretical analyses and extensive experiments to
demonstrate the effectiveness of our method, and compared with state-of-the-art
algorithms, it shows superiority in diversity, quality, and efficiency.
