Bayesian Learning in Undirected Graphical Models: Approximate MCMC
  algorithms
  Bayesian learning in undirected graphical models|computing posterior
distributions over parameters and predictive quantities is exceptionally
difficult. We conjecture that for general undirected models, there are no
tractable MCMC (Markov Chain Monte Carlo) schemes giving the correct
equilibrium distribution over parameters. While this intractability, due to the
partition function, is familiar to those performing parameter optimisation,
Bayesian learning of posterior distributions over undirected model parameters
has been unexplored and poses novel challenges. we propose several approximate
MCMC schemes and test on fully observed binary models (Boltzmann machines) for
a small coronary heart disease data set and larger artificial systems. While
approximations must perform well on the model, their interaction with the
sampling scheme is also important. Samplers based on variational mean- field
approximations generally performed poorly, more advanced methods using loopy
propagation, brief sampling and stochastic dynamics lead to acceptable
parameter posteriors. Finally, we demonstrate these techniques on a Markov
random field with hidden variables.
