Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical
  Optimality and Second-Order Convergence
  In this paper, we consider the estimation of a low Tucker rank tensor from a
number of noisy linear measurements. The general problem covers many specific
examples arising from applications, including tensor regression, tensor
completion, and tensor PCA/SVD. We propose a Riemannian Gauss-Newton (RGN)
method with fast implementations for low Tucker rank tensor estimation.
Different from the generic (super)linear convergence guarantee of RGN in the
literature, we prove the first quadratic convergence guarantee of RGN for
low-rank tensor estimation under some mild conditions. A deterministic
estimation error lower bound, which matches the upper bound, is provided that
demonstrates the statistical optimality of RGN. The merit of RGN is illustrated
through two machine learning applications: tensor regression and tensor SVD.
Finally, we provide the simulation results to corroborate our theoretical
findings.
