G-PATE: Scalable Differentially Private Data Generator via Private
  Aggregation of Teacher Discriminators
  Recent advances in machine learning have largely benefited from the massive
accessible training data. However, large-scale data sharing has raised great
privacy concerns. In this work, we propose a novel privacy-preserving data
Generative model based on the PATE framework (G-PATE), aiming to train a
scalable differentially private data generator that preserves high generated
data utility. Our approach leverages generative adversarial nets to generate
data, combined with private aggregation among different discriminators to
ensure strong privacy guarantees. Compared to existing approaches, G-PATE
significantly improves the use of privacy budgets. In particular, we train a
student data generator with an ensemble of teacher discriminators and propose a
novel private gradient aggregation mechanism to ensure differential privacy on
all information that flows from teacher discriminators to the student
generator. In addition, with random projection and gradient discretization, the
proposed gradient aggregation mechanism is able to effectively deal with
high-dimensional gradient vectors. Theoretically, we prove that G-PATE ensures
differential privacy for the data generator. Empirically, we demonstrate the
superiority of G-PATE over prior work through extensive experiments. We show
that G-PATE is the first work being able to generate high-dimensional image
data with high data utility under limited privacy budgets ($\epsilon \le 1$).
Our code is available at https://github.com/AI-secure/G-PATE.
