Conditional Expectation Propagation
  Expectation propagation (EP) is a powerful approximate inference algorithm.
However, a critical barrier in applying EP is that the moment matching in
message updates can be intractable. Handcrafting approximations is usually
tricky, and lacks generalizability. Importance sampling is very expensive.
While Laplace propagation provides a good solution, it has to run numerical
optimizations to find Laplace approximations in every update, which is still
quite inefficient. To overcome these practical barriers, we propose conditional
expectation propagation (CEP) that performs conditional moment matching given
the variables outside each message, and then takes expectation w.r.t the
approximate posterior of these variables. The conditional moments are often
analytical and much easier to derive. In the most general case, we can use
(fully) factorized messages to represent the conditional moments by quadrature
formulas. We then compute the expectation of the conditional moments via Taylor
approximations when necessary. In this way, our algorithm can always conduct
efficient, analytical fixed point iterations. Experiments on several popular
models for which standard EP is available or unavailable demonstrate the
advantages of CEP in both inference quality and computational efficiency.
