Practical Blind Membership Inference Attack via Differential Comparisons
  Membership inference (MI) attacks affect user privacy by inferring whether
given data samples have been used to train a target learning model, e.g., a
deep neural network. There are two types of MI attacks in the literature, i.e.,
these with and without shadow models. The success of the former heavily depends
on the quality of the shadow model, i.e., the transferability between the
shadow and the target; the latter, given only blackbox probing access to the
target model, cannot make an effective inference of unknowns, compared with MI
attacks using shadow models, due to the insufficient number of qualified
samples labeled with ground truth membership information.
  In this paper, we propose an MI attack, called BlindMI, which probes the
target model and extracts membership semantics via a novel approach, called
differential comparison. The high-level idea is that BlindMI first generates a
dataset with nonmembers via transforming existing samples into new samples, and
then differentially moves samples from a target dataset to the generated,
non-member set in an iterative manner. If the differential move of a sample
increases the set distance, BlindMI considers the sample as non-member and vice
versa.
  BlindMI was evaluated by comparing it with state-of-the-art MI attack
algorithms. Our evaluation shows that BlindMI improves F1-score by nearly 20%
when compared to state-of-the-art on some datasets, such as Purchase-50 and
Birds-200, in the blind setting where the adversary does not know the target
model's architecture and the target dataset's ground truth labels. We also show
that BlindMI can defeat state-of-the-art defenses.
