Collaborative Distillation for Ultra-Resolution Universal Style Transfer
  Universal style transfer methods typically leverage rich representations from
deep Convolutional Neural Network (CNN) models (e.g., VGG-19) pre-trained on
large collections of images. Despite the effectiveness, its application is
heavily constrained by the large model size to handle ultra-resolution images
given limited memory. In this work, we present a new knowledge distillation
method (named Collaborative Distillation) for encoder-decoder based neural
style transfer to reduce the convolutional filters. The main idea is
underpinned by a finding that the encoder-decoder pairs construct an exclusive
collaborative relationship, which is regarded as a new kind of knowledge for
style transfer models. Moreover, to overcome the feature size mismatch when
applying collaborative distillation, a linear embedding loss is introduced to
drive the student network to learn a linear embedding of the teacher's
features. Extensive experiments show the effectiveness of our method when
applied to different universal style transfer approaches (WCT and AdaIN), even
if the model size is reduced by 15.5 times. Especially, on WCT with the
compressed models, we achieve ultra-resolution (over 40 megapixels) universal
style transfer on a 12GB GPU for the first time. Further experiments on
optimization-based stylization scheme show the generality of our algorithm on
different stylization paradigms. Our code and trained models are available at
https://github.com/mingsun-tse/collaborative-distillation.
