StolenEncoder: Stealing Pre-trained Encoders
  Pre-trained encoders are general-purpose feature extractors that can be used
for many downstream tasks. Recent progress in self-supervised learning can
pre-train highly effective encoders using a large volume of unlabeled data,
leading to the emerging encoder as a service (EaaS). A pre-trained encoder may
be deemed confidential because its training often requires lots of data and
computation resources as well as its public release may facilitate misuse of
AI, e.g., for deepfakes generation. In this paper, we propose the first attack
called StolenEncoder to steal pre-trained image encoders. We evaluate
StolenEncoder on multiple target encoders pre-trained by ourselves and three
real-world target encoders including the ImageNet encoder pre-trained by
Google, CLIP encoder pre-trained by OpenAI, and Clarifai's General Embedding
encoder deployed as a paid EaaS. Our results show that the encoders stolen by
StolenEncoder have similar functionality with the target encoders. In
particular, the downstream classifiers built upon a target encoder and a stolen
encoder have similar accuracy. Moreover, stealing a target encoder using
StolenEncoder requires much less data and computation resources than
pre-training it from scratch. We also explore three defenses that perturb
feature vectors produced by a target encoder. Our evaluation shows that these
defenses are not enough to mitigate StolenEncoder.
