Protection Against Reconstruction and Its Applications in Private
  Federated Learning
  In large-scale statistical learning, data collection and model fitting are
moving increasingly toward peripheral devices---phones, watches, fitness
trackers---away from centralized data collection. Concomitant with this rise in
decentralized data are increasing challenges of maintaining privacy while
allowing enough information to fit accurate, useful statistical models. This
motivates local notions of privacy---most significantly, local differential
privacy, which provides strong protections against sensitive data
disclosures---where data is obfuscated before a statistician or learner can
even observe it, providing strong protections to individuals' data. Yet local
privacy as traditionally employed may prove too stringent for practical use,
especially in modern high-dimensional statistical and machine learning
problems. Consequently, we revisit the types of disclosures and adversaries
against which we provide protections, considering adversaries with limited
prior information and ensuring that with high probability, ensuring they cannot
reconstruct an individual's data within useful tolerances. By reconceptualizing
these protections, we allow more useful data release---large privacy parameters
in local differential privacy---and we design new (minimax) optimal locally
differentially private mechanisms for statistical learning problems for
\emph{all} privacy levels. We thus present practicable approaches to
large-scale locally private model training that were previously impossible,
showing theoretically and empirically that we can fit large-scale image
classification and language models with little degradation in utility.
