Multimeasurement Generative Models
  We formally map the problem of sampling from an unknown distribution with a
density in $\mathbb{R}^d$ to the problem of learning and sampling a smoother
density in $\mathbb{R}^{Md}$ obtained by convolution with a fixed factorial
kernel: the new density is referred to as M-density and the kernel as
multimeasurement noise model (MNM). The M-density in $\mathbb{R}^{Md}$ is
smoother than the original density in $\mathbb{R}^d$, easier to learn and
sample from, yet for large $M$ the two problems are mathematically equivalent
since clean data can be estimated exactly given a multimeasurement noisy
observation using the Bayes estimator. To formulate the problem, we derive the
Bayes estimator for Poisson and Gaussian MNMs in closed form in terms of the
unnormalized M-density. This leads to a simple least-squares objective for
learning parametric energy and score functions. We present various
parametrization schemes of interest including one in which studying Gaussian
M-densities directly leads to multidenoising autoencoders--this is the first
theoretical connection made between denoising autoencoders and empirical Bayes
in the literature. Samples in $\mathbb{R}^d$ are obtained by walk-jump sampling
(Saremi & Hyvarinen, 2019) via underdamped Langevin MCMC (walk) to sample from
M-density and the multimeasurement Bayes estimation (jump). We study
permutation invariant Gaussian M-densities on MNIST, CIFAR-10, and FFHQ-256
datasets, and demonstrate the effectiveness of this framework for realizing
fast-mixing stable Markov chains in high dimensions.
