Defending Model Inversion and Membership Inference Attacks via
  Prediction Purification
  Neural networks are susceptible to data inference attacks such as the model
inversion attack and the membership inference attack, where the attacker could
infer the reconstruction and the membership of a data sample from the
confidence scores predicted by the target classifier. In this paper, we propose
a unified approach, namely purification framework, to defend data inference
attacks. It purifies the confidence score vectors predicted by the target
classifier by reducing their dispersion. The purifier can be further
specialized in defending a particular attack via adversarial learning. We
evaluate our approach on benchmark datasets and classifiers. We show that when
the purifier is dedicated to one attack, it naturally defends the other one,
which empirically demonstrates the connection between the two attacks. The
purifier can effectively defend both attacks. For example, it can reduce the
membership inference accuracy by up to 15% and increase the model inversion
error by a factor of up to 4. Besides, it incurs less than 0.4% classification
accuracy drop and less than 5.5% distortion to the confidence scores.
