Sum-of-squares meets square loss: Fast rates for agnostic tensor
  completion
  We study tensor completion in the agnostic setting. In the classical tensor
completion problem, we receive $n$ entries of an unknown rank-$r$ tensor and
wish to exactly complete the remaining entries. In agnostic tensor completion,
we make no assumption on the rank of the unknown tensor, but attempt to predict
unknown entries as well as the best rank-$r$ tensor.
  For agnostic learning of third-order tensors with the square loss, we give
the first polynomial time algorithm that obtains a "fast" (i.e., $O(1/n)$-type)
rate improving over the rate obtained by reduction to matrix completion. Our
prediction error rate to compete with the best $d\times{}d\times{}d$ tensor of
rank-$r$ is $\tilde{O}(r^{2}d^{3/2}/n)$. We also obtain an exact oracle
inequality that trades off estimation and approximation error.
  Our algorithm is based on the degree-six sum-of-squares relaxation of the
tensor nuclear norm. The key feature of our analysis is to show that a certain
characterization for the subgradient of the tensor nuclear norm can be encoded
in the sum-of-squares proof system. This unlocks the standard toolbox for
localization of empirical processes under the square loss, and allows us to
establish restricted eigenvalue-type guarantees for various tensor regression
models, with tensor completion as a special case. The new analysis of the
relaxation complements Barak and Moitra (2016), who gave slow rates for
agnostic tensor completion, and Potechin and Steurer (2017), who gave exact
recovery guarantees for the noiseless setting. Our techniques are
user-friendly, and we anticipate that they will find use elsewhere.
