Flexibly Regularized Mixture Models and Application to Image
  Segmentation
  Probabilistic finite mixture models are widely used for unsupervised
clustering. These models can often be improved by adapting them to the topology
of the data. For instance, in order to classify spatially adjacent data points
similarly, it is common to introduce a Laplacian constraint on the posterior
probability that each data point belongs to a class. Alternatively, the mixing
probabilities can be treated as free parameters, while assuming Gauss-Markov or
more complex priors to regularize those mixing probabilities. However, these
approaches are constrained by the shape of the prior and often lead to
complicated or intractable inference. Here, we propose a new parametrization of
the Dirichlet distribution to flexibly regularize the mixing probabilities of
over-parametrized mixture distributions. Using the Expectation-Maximization
algorithm, we show that our approach allows us to define any linear update rule
for the mixing probabilities, including spatial smoothing regularization as a
special case. We then show that this flexible design can be extended to share
class information between multiple mixture models. We apply our algorithm to
artificial and natural image segmentation tasks, and we provide quantitative
and qualitative comparison of the performance of Gaussian and Student-t
mixtures on the Berkeley Segmentation Dataset. We also demonstrate how to
propagate class information across the layers of deep convolutional neural
networks in a probabilistically optimal way, suggesting a new interpretation
for feedback signals in biological visual systems. Our flexible approach can be
easily generalized to adapt probabilistic mixture models to arbitrary data
topologies.
