Loopy Belief Propagation for Approximate Inference: An Empirical Study
  Recently, researchers have demonstrated that loopy belief propagation - the
use of Pearls polytree algorithm IN a Bayesian network WITH loops OF error-
correcting codes.The most dramatic instance OF this IS the near Shannon - limit
performance OF Turbo Codes codes whose decoding algorithm IS equivalent TO
loopy belief propagation IN a chain - structured Bayesian network. IN this
paper we ask : IS there something special about the error - correcting code
context, OR does loopy propagation WORK AS an approximate inference schemeIN a
more general setting? We compare the marginals computed using loopy propagation
TO the exact ones IN four Bayesian network architectures, including two real -
world networks : ALARM AND QMR.We find that the loopy beliefs often converge
AND WHEN they do, they give a good approximation TO the correct
marginals.However,ON the QMR network, the loopy beliefs oscillated AND had no
obvious relationship TO the correct posteriors. We present SOME initial
investigations INTO the cause OF these oscillations, AND show that SOME simple
methods OF preventing them lead TO the wrong results.
