Augmented Tensor Decomposition with Stochastic Optimization
  Tensor decompositions are powerful tools for dimensionality reduction and
feature interpretation of multidimensional data such as signals. Existing
tensor decomposition objectives (e.g., Frobenius norm) are designed for fitting
raw data under statistical assumptions, which may not align with downstream
classification tasks. Also, real-world tensor data are usually high-ordered and
have large dimensions with millions or billions of entries. Thus, it is
expensive to decompose the whole tensor with traditional algorithms. In
practice, raw tensor data also contains redundant information while data
augmentation techniques may be used to smooth out noise in samples. This paper
addresses the above challenges by proposing augmented tensor decomposition
(ATD), which effectively incorporates data augmentations to boost downstream
classification. To reduce the memory footprint of the decomposition, we propose
a stochastic algorithm that updates the factor matrices in a batch fashion. We
evaluate ATD on multiple signal datasets. It shows comparable or better
performance (e.g., up to 15% in accuracy) over self-supervised and autoencoder
baselines with less than 5% of model parameters, achieves 0.6% ~ 1.3% accuracy
gain over other tensor-based baselines, and reduces the memory footprint by 9X
when compared to standard tensor decomposition algorithms.
