A new perspective on probabilistic image modeling
  We present the Deep Convolutional Gaussian Mixture Model (DCGMM), a new
probabilistic approach for image modeling capable of density estimation,
sampling and tractable inference. DCGMM instances exhibit a CNN-like layered
structure, in which the principal building blocks are convolutional Gaussian
Mixture (cGMM) layers. A key innovation w.r.t. related models like sum-product
networks (SPNs) and probabilistic circuits (PCs) is that each cGMM layer
optimizes an independent loss function and therefore has an independent
probabilistic interpretation. This modular approach permits intervening
transformation layers to harness the full spectrum of (potentially
non-invertible) mappings available to CNNs, e.g., max-pooling or
half-convolutions. DCGMM sampling and inference are realized by a deep chain of
hierarchical priors, where a sample generated by a given cGMM layer defines the
parameters of sampling in the next-lower cGMM layer. For sampling through
non-invertible transformation layers, we introduce a new gradient-based
sharpening technique that exploits redundancy (overlap) in, e.g.,
half-convolutions. DCGMMs can be trained end-to-end by SGD from random initial
conditions, much like CNNs. We show that DCGMMs compare favorably to several
recent PC and SPN models in terms of inference, classification and sampling,
the latter particularly for challenging datasets such as SVHN. We provide a
public TF2 implementation.
