Inferring Parameters and Structure of Latent Variable Models by
  Variational Bayes
  Current methods for learning graphical models with latent variables and a
fixed structure estimate optimal values for the model parameters. Whereas this
approach usually produces overfitting and suboptimal generalization
performance, carrying out the Bayesian program of computing the full posterior
distributions over the parameters remains a difficult problem. Moreover,
learning the structure of models with latent variables, for which the Bayesian
approach is crucial, is yet a harder problem. In this paper I present the
Variational Bayes framework, which provides a solution to these problems. This
approach approximates full posterior distributions over model parameters and
structures, as well as latent variables, in an analytical manner without
resorting to sampling methods. Unlike in the Laplace approximation, these
posteriors are generally non-Gaussian and no Hessian needs to be computed. The
resulting algorithm generalizes the standard Expectation Maximization
algorithm, and its convergence is guaranteed. I demonstrate that this algorithm
can be applied to a large class of models in several domains, including
unsupervised clustering and blind source separation.
