cpSGD: Communication-efficient and differentially-private distributed
  SGD
  Distributed stochastic gradient descent is an important subroutine in
distributed learning. A setting of particular interest is when the clients are
mobile devices, where two important concerns are communication efficiency and
the privacy of the clients. Several recent works have focused on reducing the
communication cost or introducing privacy guarantees, but none of the proposed
communication efficient methods are known to be privacy preserving and none of
the known privacy mechanisms are known to be communication efficient. To this
end, we study algorithms that achieve both communication efficiency and
differential privacy. For $d$ variables and $n \approx d$ clients, the proposed
method uses $O(\log \log(nd))$ bits of communication per client per coordinate
and ensures constant privacy.
  We also extend and improve previous analysis of the \emph{Binomial mechanism}
showing that it achieves nearly the same utility as the Gaussian mechanism,
while requiring fewer representation bits, which can be of independent
interest.
