A Generative Model for Texture Synthesis based on Optimal Transport
  between Feature Distributions
  We propose GOTEX, a general framework for texture synthesis by optimization
that constrains the statistical distribution of local features. While our model
encompasses several existing texture models, we focus on the case where the
comparison between feature distributions relies on optimal transport distances.
We show that the semi-dual formulation of optimal transport allows to control
the distribution of various possible features, even if these features live in a
high-dimensional space. We then study the resulting minimax optimization
problem, which corresponds to a Wasserstein generative model, for which the
inner concave maximization problem can be solved with standard stochastic
gradient methods. The alternate optimization algorithm is shown to be versatile
in terms of applications, features and architecture; in particular it allows to
produce high-quality synthesized textures with different sets of features. We
analyze the results obtained by constraining the distribution of patches or the
distribution of responses to a pre-learned VGG neural network. We show that the
patch representation can retrieve the desired textural aspect in a more precise
manner. We also provide a detailed comparison with state-of-the-art texture
synthesis methods. The GOTEX model based on patch features is also adapted to
texture inpainting and texture interpolation. Finally, we show how to use our
framework to learn a feed-forward neural network that can synthesize on-the-fly
new textures of arbitrary size in a very fast manner. Experimental results and
comparisons with the mainstream methods from the literature illustrate the
relevance of the generative models learned with GOTEX.
