are simply 1 =‘$_{ii}$ , where the ‘$_{ii}$ ’s form the diagonal of L$_{h}$ . In practice we do the GLYPH<22>$_{h}$ and 6$_{h}$ updates in a single step by optimizing jointly over GLYPH<22>$_{h}$ and L$_{h}$ , which remains a convex problem. In the M-step, we maximize (25) over GLYPH<16> and GLYPH<127> . Identifying the terms which depend on GLYPH<16> , we recognize the usual Gaussian mean estimation problem. Further, (25) is easily seen to be concave in GLYPH<127> GLYPH<0> $^{1}$, with a closed-form solution of the corresponding first-order condition. We obtain the M-step updates Here ̂ Cov .GLYPH<22>$_{GLYPH<1>}$ / is the empirical covariance of the GLYPH<22>$_{h}$ vectors. In the fully Bayesian MML model, GLYPH<16> and GLYPH<127> have prior distributions, with corresponding variational factors given in (14). The ELBO in this case has the same form as (20), with two differences. First, H . q / contains two new terms Second, there are two new cross-entropy terms Also, the middle term of (20) changes in the fully Bayesian case, because GLYPH<16> and GLYPH<127> are now averaged over rather than treated as constants. Using known formulas for normal and Wishart entropies, the two new entropy terms 23 A.4 Variational hierarchical Bayes A.3 Empirical Bayes M-step O GLYPH<16> 1 H H ∑ h D 1 GLYPH<22>$_{h}$ ; O GLYPH<127> 1 H H ∑ h D 1 6$_{h}$ C ̂ Cov .GLYPH<22>$_{GLYPH<1>}$ / : (30) H $^{(}$q$^{(}$GLYPH<16> j GLYPH<22>$_{GLYPH<16>}$ ; 6$_{GLYPH<16>}$ $^{))}$C H $^{(}$q$^{(}$GLYPH<127> j 7 GLYPH<0> $^{1}$; ! )) : (31) ❊ q log p .GLYPH<16> j GLYPH<12>$_{0}$ ; GLYPH<127>$_{0}$ / C ❊ q log p .GLYPH<127> j S ; GLYPH<23> / : (32)