Investigating Membership Inference Attacks under Data Dependencies
  Training machine learning models on privacy-sensitive data has become a
popular practice, driving innovation in ever-expanding fields. This has opened
the door to new attacks that can have serious privacy implications. One such
attack, the Membership Inference Attack (MIA), exposes whether or not a
particular data point was used to train a model. A growing body of literature
uses Differentially Private (DP) training algorithms as a defence against such
attacks. However, these works evaluate the defence under the restrictive
assumption that all members of the training set, as well as non-members, are
independent and identically distributed. This assumption does not hold for many
real-world use cases in the literature. Motivated by this, we evaluate
membership inference with statistical dependencies among samples and explain
why DP does not provide meaningful protection (the privacy parameter $\epsilon$
scales with the training set size $n$) in this more general case. We conduct a
series of empirical evaluations with off-the-shelf MIAs using training sets
built from real-world data showing different types of dependencies among
samples. Our results reveal that training set dependencies can severely
increase the performance of MIAs, and therefore assuming that data samples are
statistically independent can significantly underestimate the performance of
MIAs.
