Effects of Differential Privacy and Data Skewness on Membership
  Inference Vulnerability
  Membership inference attacks seek to infer the membership of individual
training instances of a privately trained model. This paper presents a
membership privacy analysis and evaluation system, called MPLens, with three
unique contributions. First, through MPLens, we demonstrate how membership
inference attack methods can be leveraged in adversarial machine learning.
Second, through MPLens, we highlight how the vulnerability of pre-trained
models under membership inference attack is not uniform across all classes,
particularly when the training data itself is skewed. We show that risk from
membership inference attacks is routinely increased when models use skewed
training data. Finally, we investigate the effectiveness of differential
privacy as a mitigation technique against membership inference attacks. We
discuss the trade-offs of implementing such a mitigation strategy with respect
to the model complexity, the learning task complexity, the dataset complexity
and the privacy parameter settings. Our empirical results reveal that (1)
minority groups within skewed datasets display increased risk for membership
inference and (2) differential privacy presents many challenging trade-offs as
a mitigation technique to membership inference risk.
