Fast and Accurate Randomized Algorithms for Low-rank Tensor
  Decompositions
  Low-rank Tucker and CP tensor decompositions are powerful tools in data
analytics. The widely used alternating least squares (ALS) method, which solves
a sequence of over-determined least squares subproblems, is costly for large
and sparse tensors. We propose a fast and accurate sketched ALS algorithm for
Tucker decomposition, which solves a sequence of sketched rank-constrained
linear least squares subproblems. Theoretical sketch size upper bounds are
provided to achieve $O(\epsilon)$ relative error for each subproblem with two
sketching techniques, TensorSketch and leverage score sampling. Experimental
results show that this new ALS algorithm, combined with a new initialization
scheme based on randomized range finder, yields up to $22.0\%$ relative
decomposition residual improvement compared to the state-of-the-art sketched
randomized algorithm for Tucker decomposition of various synthetic and real
datasets. This Tucker-ALS algorithm is further used to accelerate CP
decomposition, by using randomized Tucker compression followed by CP
decomposition of the Tucker core tensor. Experimental results show that this
algorithm not only converges faster, but also yields more accurate CP
decompositions.
