The numerator in both cases is the joint density of latent and observed variables, computed by multiplying together the densities defined in the model hierarchy (3)–(5). The integrals appearing in these posterior densities have no closed form. As a conse- quence, exact inference is intractable. Variational inference is a deterministic alternative to the MCMC methods usually applied to this problem (Rossi et al. 2005). A variational algo- rithm selects from a pre-specified family of distributions Q the best approximation to the true posterior distribution. We define Q so that all of its members permit tractable probabil- ity calculations. Then, wherever we need the true posterior, such as for an expectation, we use the approximating variational distribution instead. This plug-in idea underlies MCMC methods as well – in place of the true posterior, we substitute the empirical distribution of the MCMC posterior draws. In this section we give an overview of a variational algorithm for approximate empirical Bayes estimation in the MML model. Appendix A fills in the details. We first specify a family of approximating distributions Q : D f q .GLYPH<12>$_{1:}$$_{H}$ j GLYPH<21>/ : GLYPH<21> 2 3 g for the true pos- terior distribution (6). Since this posterior factors over h , we take Q to be a family of factored distributions as well, so that q .GLYPH<12>$_{1:}$$_{H}$ j GLYPH<21>/ : D ∏ h q .GLYPH<12>$_{h}$ j GLYPH<21>$_{h}$ / . In particular, each factor q .GLYPH<12>$_{h}$ j GLYPH<21>$_{h}$ / is a K -variate normal density, with mean GLYPH<22>$_{h}$ and covariance matrix 6$_{h}$ . For the particular data set at hand, we want to find q .GLYPH<12>$_{1:}$$_{H}$ j GLYPH<21> $^{GLYPH<3>}$/ , the best approximation in Q to the posterior distribution p .GLYPH<12>$_{1:}$$_{H}$ j D ; GLYPH<16> ; GLYPH<127>/ . To make the idea of a best approxi- mation precise, we measure discrepancy with the Kullback-Leibler (KL) divergence (also called the relative entropy). Shortening q .GLYPH<12>$_{1:}$$_{H}$ j GLYPH<21>/ to q$_{GLYPH<21>}$ , the optimal variational parame- ters are given by 7 GLYPH<21> GLYPH<3> D argmin GLYPH<21> 2 3 KL $^{[}$q$_{GLYPH<21>}$ jj p $^{]}$: (8) 3.1 Variational empirical Bayes