Note the similarity of this gradient to the gradient from an L$_{2}$ -regularized multiple logistic regression: it consists of a contribution from the regularizer (the left-hand term), plus a residual-weighted sum of covariate vectors. Abbreviating w.GLYPH<22>$_{h}$; 6$_{h}$; x$_{ht}$ / to w$_{ht}$ , an argu- ment using matrix differentials (Magnus and Neudecker 2007) gives the Hessian The 6$_{h}$ coordinate update is harder, because we need to insure that 6$_{h}$ GLYPH<23> 0. Using a reformulation, we can avoid making the constraint explicit, which would complicate the optimization. Let 6$_{h}$ D L$_{h}$ L > h for a lower-triangular matrix L$_{h}$ . Since 6$_{h}$ GLYPH<23> 0, one such L$_{h}$ always exists—the Cholesky factor. We replace each 6$_{h}$ in Q L with 6$_{h}$. L$_{h}$ / : D L$_{h}$ L > h , and optimize over the unconstrained set of lower-triangular matrices L$_{h}$ . The objective function (25) remains concave in L$_{h}$ . To see this, compare the terms depending on 6$_{h}$ D L$_{h}$ L > h to the function studied in Appendix C. We now give the gradient with respect to L$_{h}$ . Standard matrix differentiation of (25) leads to the 6$_{h}$ gradient Again using matrix differentials and the Cauchy invariance rule, it is not hard to show that the gradient with respect to L$_{h}$ is Note that this is the gradient with respect to a dense matrix L$_{h}$ . Since we optimize over lower-triangular matrices, i.e. vech( L$_{h}$ ), we need only use the lower triangular of the gra- dient. This is convenient for the term L GLYPH<0>> h : it is upper-triangular, so its lower triangle is a diagonal matrix. Furthermore, from a standard result of linear algebra, the diagonal entries 22 @ Q L @GLYPH<22>$_{h}$@GLYPH<22> > h D GLYPH<0> GLYPH<127> GLYPH<0> 1 GLYPH<0> T$_{h}$ ∑ t D 1 $^{[}$x > ht diag f w$_{ht}$ g x$_{ht}$ GLYPH<0> . x > $_{ht}$w$_{ht}$ /. x > $_{ht}$w$_{ht}$ / > ] : (27) @ Q L @6$_{h}$ D 1 2 [ 6 GLYPH<0> 1 h GLYPH<0> GLYPH<127> GLYPH<0> 1 GLYPH<0> T$_{h}$ ∑ t D 1 x > $_{ht}$diag f w$_{ht}$ g x$_{ht}$$^{]}$ : (28) @ Q L @ L$_{h}$ D 2$^{(}$ @ Q L @6$_{h}$$^{)}$ L$_{h}$ D L GLYPH<0>> h GLYPH<0>$^{(}$ GLYPH<127> GLYPH<0> 1 C T$_{h}$ ∑ t D 1 x > $_{ht}$diag f w$_{ht}$ g x$_{ht}$$^{)}$ L$_{h}$ : (29)