Distributed Differentially-Private Algorithms for Matrix and Tensor
  Factorization
  In many signal processing and machine learning applications, datasets
containing private information are held at different locations, requiring the
development of distributed privacy-preserving algorithms. Tensor and matrix
factorizations are key components of many processing pipelines. In the
distributed setting, differentially private algorithms suffer because they
introduce noise to guarantee privacy. This paper designs new and improved
distributed and differentially private algorithms for two popular matrix and
tensor factorization methods: principal component analysis (PCA) and orthogonal
tensor decomposition (OTD). The new algorithms employ a correlated noise design
scheme to alleviate the effects of noise and can achieve the same noise level
as the centralized scenario. Experiments on synthetic and real data illustrate
the regimes in which the correlated noise allows performance matching with the
centralized setting, outperforming previous methods and demonstrating that
meaningful utility is possible while guaranteeing differential privacy.
