Locally Private Learning without Interaction Requires Separation
  We consider learning under the constraint of local differential privacy
(LDP). For many learning problems known efficient algorithms in this model
require many rounds of communication between the server and the clients holding
the data points. Yet multi-round protocols are prohibitively slow in practice
due to network latency and, as a result, currently deployed large-scale systems
are limited to a single round. Despite significant research interest, very
little is known about which learning problems can be solved by such
non-interactive systems. The only lower bound we are aware of is for PAC
learning an artificial class of functions with respect to a uniform
distribution (Kasiviswanathan et al. 2011).
  We show that the margin complexity of a class of Boolean functions is a lower
bound on the complexity of any non-interactive LDP algorithm for
distribution-independent PAC learning of the class. In particular, the classes
of linear separators and decision lists require exponential number of samples
to learn non-interactively even though they can be learned in polynomial time
by an interactive LDP algorithm. This gives the first example of a natural
problem that is significantly harder to solve without interaction and also
resolves an open problem of Kasiviswanathan et al. (2011). We complement this
lower bound with a new efficient learning algorithm whose complexity is
polynomial in the margin complexity of the class. Our algorithm is
non-interactive on labeled samples but still needs interactive access to
unlabeled samples. All of our results also apply to the statistical query model
and any model in which the number of bits communicated about each data point is
constrained.
