Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC
  The damped Gauss-Newton (dGN) algorithm for CANDECOMP/PARAFAC (CP)
decomposition can handle the challenges of collinearity of factors and
different magnitudes of factors; nevertheless, for factorization of an $N$-D
tensor of size $I_1\times I_N$ with rank $R$, the algorithm is computationally
demanding due to construction of large approximate Hessian of size $(RT \times
RT)$ and its inversion where $T = \sum_n I_n$. In this paper, we propose a fast
implementation of the dGN algorithm which is based on novel expressions of the
inverse approximate Hessian in block form. The new implementation has lower
computational complexity, besides computation of the gradient (this part is
common to both methods), requiring the inversion of a matrix of size
$NR^2\times NR^2$, which is much smaller than the whole approximate Hessian, if
$T \gg NR$. In addition, the implementation has lower memory requirements,
because neither the Hessian nor its inverse never need to be stored in their
entirety. A variant of the algorithm working with complex valued data is
proposed as well. Complexity and performance of the proposed algorithm is
compared with those of dGN and ALS with line search on examples of difficult
benchmark tensors.
