Copula & Marginal Flows: Disentangling the Marginal from its Joint
  Deep generative networks such as GANs and normalizing flows flourish in the
context of high-dimensional tasks such as image generation. However, so far
exact modeling or extrapolation of distributional properties such as the tail
asymptotics generated by a generative network is not available. In this paper,
we address this issue for the first time in the deep learning literature by
making two novel contributions. First, we derive upper bounds for the tails
that can be expressed by a generative network and demonstrate Lp-space related
properties. There we show specifically that in various situations an optimal
generative network does not exist. Second, we introduce and propose copula and
marginal generative flows (CM flows) which allow for an exact modeling of the
tail and any prior assumption on the CDF up to an approximation of the uniform
distribution. Our numerical results support the use of CM flows.
