Stochastic Gradients for Large-Scale Tensor Decomposition
  Tensor decomposition is a well-known tool for multiway data analysis. This
work proposes using stochastic gradients for efficient generalized canonical
polyadic (GCP) tensor decomposition of large-scale tensors. GCP tensor
decomposition is a recently proposed version of tensor decomposition that
allows for a variety of loss functions such as Bernoulli loss for binary data
or Huber loss for robust estimation. The stochastic gradient is formed from
randomly sampled elements of the tensor and is efficient because it can be
computed using the sparse matricized-tensor-times-Khatri-Rao product (MTTKRP)
tensor kernel. For dense tensors, we simply use uniform sampling. For sparse
tensors, we propose two types of stratified sampling that give precedence to
sampling nonzeros. Numerical results demonstrate the advantages of the proposed
approach and its scalability to large-scale problems.
