Generative Models for Effective ML on Private, Decentralized Datasets
  To improve real-world applications of machine learning, experienced modelers
develop intuition about their datasets, their models, and how the two interact.
Manual inspection of raw data - of representative samples, of outliers, of
misclassifications - is an essential tool in a) identifying and fixing problems
in the data, b) generating new modeling hypotheses, and c) assigning or
refining human-provided labels. However, manual data inspection is problematic
for privacy sensitive datasets, such as those representing the behavior of
real-world individuals. Furthermore, manual data inspection is impossible in
the increasingly important setting of federated learning, where raw examples
are stored at the edge and the modeler may only access aggregated outputs such
as metrics or model parameters. This paper demonstrates that generative models
- trained using federated methods and with formal differential privacy
guarantees - can be used effectively to debug many commonly occurring data
issues even when the data cannot be directly inspected. We explore these
methods in applications to text with differentially private federated RNNs and
to images using a novel algorithm for differentially private federated GANs.
