Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be
  Secretly Coded into the Classifiers' Outputs
  It is known that deep neural networks, trained for the classification of
non-sensitive target attributes, can reveal sensitive attributes of their input
data through internal representations extracted by the classifier. We take a
step forward and show that deep classifiers can be trained to secretly encode a
sensitive attribute of their input data into the classifier's outputs for the
target attribute, at inference time. Our proposed attack works even if users
have a full white-box view of the classifier, can keep all internal
representations hidden, and only release the classifier's estimations for the
target attribute. We introduce an information-theoretical formulation for such
attacks and present efficient empirical implementations for training
honest-but-curious (HBC) classifiers: classifiers that can be accurate in
predicting their target attribute, but can also exploit their outputs to
secretly encode a sensitive attribute. Our work highlights a vulnerability that
can be exploited by malicious machine learning service providers to attack
their user's privacy in several seemingly safe scenarios; such as encrypted
inferences, computations at the edge, or private knowledge distillation.
Experimental results on several attributes in two face-image datasets show that
a semi-trusted server can train classifiers that are not only perfectly honest
but also accurately curious. We conclude by showing the difficulties in
distinguishing between standard and HBC classifiers, discussing challenges in
defending against this vulnerability of deep classifiers, and enumerating
related open directions for future studies.
