In the light of feature distributions: moment matching for Neural Style
  Transfer
  Style transfer aims to render the content of a given image in the
graphical/artistic style of another image. The fundamental concept underlying
NeuralStyle Transfer (NST) is to interpret style as a distribution in the
feature space of a Convolutional Neural Network, such that a desired style can
be achieved by matching its feature distribution. We show that most current
implementations of that concept have important theoretical and practical
limitations, as they only partially align the feature distributions. We propose
a novel approach that matches the distributions more precisely, thus
reproducing the desired style more faithfully, while still being
computationally efficient. Specifically, we adapt the dual form of Central
Moment Discrepancy (CMD), as recently proposed for domain adaptation, to
minimize the difference between the target style and the feature distribution
of the output image. The dual interpretation of this metric explicitly matches
all higher-order centralized moments and is therefore a natural extension of
existing NST methods that only take into account the first and second moments.
Our experiments confirm that the strong theoretical properties also translate
to visually better style transfer, and better disentangle style from semantic
image content.
