An end-to-end Differentially Private Latent Dirichlet Allocation Using a
  Spectral Algorithm
  We provide an end-to-end differentially private spectral algorithm for
learning LDA, based on matrix/tensor decompositions, and establish theoretical
guarantees on utility/consistency of the estimated model parameters. The
spectral algorithm consists of multiple algorithmic steps, named as "{edges}",
to which noise could be injected to obtain differential privacy. We identify
\emph{subsets of edges}, named as "{configurations}", such that adding noise to
all edges in such a subset guarantees differential privacy of the end-to-end
spectral algorithm. We characterize the sensitivity of the edges with respect
to the input and thus estimate the amount of noise to be added to each edge for
any required privacy level. We then characterize the utility loss for each
configuration as a function of injected noise. Overall, by combining the
sensitivity and utility characterization, we obtain an end-to-end
differentially private spectral algorithm for LDA and identify the
corresponding configuration that outperforms others in any specific regime. We
are the first to achieve utility guarantees under the required level of
differential privacy for learning in LDA. Overall our method systematically
outperforms differentially private variational inference.
