Distributed Nonparametric Regression under Communication Constraints
  This paper studies the problem of nonparametric estimation of a smooth
function with data distributed across multiple machines. We assume an
independent sample from a white noise model is collected at each machine, and
an estimator of the underlying true function needs to be constructed at a
central machine. We place limits on the number of bits that each machine can
use to transmit information to the central machine. Our results give both
asymptotic lower bounds and matching upper bounds on the statistical risk under
various settings. We identify three regimes, depending on the relationship
among the number of machines, the size of the data available at each machine,
and the communication budget. When the communication budget is small, the
statistical risk depends solely on this communication bottleneck, regardless of
the sample size. In the regime where the communication budget is large, the
classic minimax risk in the non-distributed estimation setting is recovered. In
an intermediate regime, the statistical risk depends on both the sample size
and the communication budget.
