Enhanced Membership Inference Attacks against Machine Learning Models
  How much does a given trained model leak about each individual data record in
its training set? Membership inference attacks are used as an auditing tool to
quantify the private information that a model leaks about the individual data
points in its training set. Membership inference attacks are influenced by
different uncertainties that an attacker has to resolve about training data,
the training algorithm, and the underlying data distribution. Thus attack
success rates, of many attacks in the literature, do not precisely capture the
information leakage of models about their data, as they also reflect other
uncertainties that the attack algorithm has. In this paper, we explain the
implicit assumptions and also the simplifications made in prior work using the
framework of hypothesis testing. We also derive new attack algorithms from the
framework that can achieve a high AUC score while also highlighting the
different factors that affect their performance. Our algorithms capture a very
precise approximation of privacy loss in models, and can be used as a tool to
perform an accurate and informed estimation of privacy risk in machine learning
models. We provide a thorough empirical evaluation of our attack strategies on
various machine learning tasks and benchmark datasets.
