A general method for regularizing tensor decomposition methods via
  pseudo-data
  Tensor decomposition methods allow us to learn the parameters of latent
variable models through decomposition of low-order moments of data. A
significant limitation of these algorithms is that there exists no general
method to regularize them, and in the past regularization has mostly been
performed using bespoke modifications to the algorithms, tailored for the
particular form of the desired regularizer. We present a general method of
regularizing tensor decomposition methods which can be used for any likelihood
model that is learnable using tensor decomposition methods and any
differentiable regularization function by supplementing the training data with
pseudo-data. The pseudo-data is optimized to balance two terms: being as close
as possible to the true data and enforcing the desired regularization. On
synthetic, semi-synthetic and real data, we demonstrate that our method can
improve inference accuracy and regularize for a broad range of goals including
transfer learning, sparsity, interpretability, and orthogonality of the learned
parameters.
