Personalized PATE: Differential Privacy for Machine Learning with
  Individual Privacy Guarantees
  Applying machine learning (ML) to sensitive domains requires privacy
protection of the underlying training data through formal privacy frameworks,
such as differential privacy (DP). Yet, usually, the privacy of the training
data comes at the costs of the resulting ML models' utility. One reason for
this is that DP uses one homogeneous privacy budget epsilon for all training
data points, which has to align with the strictest privacy requirement
encountered among all data holders. In practice, different data holders might
have different privacy requirements and data points of data holders with lower
requirements could potentially contribute more information to the training
process of the ML models. To account for this possibility, we propose three
novel methods that extend the DP framework Private Aggregation of Teacher
Ensembles (PATE) to support training an ML model with different personalized
privacy guarantees within the training data. We formally describe the methods,
provide theoretical analyses of their privacy bounds, and experimentally
evaluate their effect on the final model's utility at the example of the MNIST
and Adult income datasets. Our experiments show that our personalized privacy
methods yield higher accuracy models than the non-personalized baseline.
Thereby, our methods can improve the privacy-utility trade-off in scenarios in
which different data holders consent to contribute their sensitive data at
different privacy levels.
