Understanding the Security of Deepfake Detection
  Deepfakes pose growing challenges to the trust of information on the
Internet. Thus, detecting deepfakes has attracted increasing attentions from
both academia and industry. State-of-the-art deepfake detection methods consist
of two key components, i.e., face extractor and face classifier, which extract
the face region in an image and classify it to be real/fake, respectively.
Existing studies mainly focused on improving the detection performance in
non-adversarial settings, leaving security of deepfake detection in adversarial
settings largely unexplored. In this work, we aim to bridge the gap. In
particular, we perform a systematic measurement study to understand the
security of the state-of-the-art deepfake detection methods in adversarial
settings. We use two large-scale public deepfakes data sources including
FaceForensics++ and Facebook Deepfake Detection Challenge, where the deepfakes
are fake face images; and we train state-of-the-art deepfake detection methods.
These detection methods can achieve 0.94--0.99 accuracies in non-adversarial
settings on these datasets. However, our measurement results uncover multiple
security limitations of the deepfake detection methods in adversarial settings.
First, we find that an attacker can evade a face extractor, i.e., the face
extractor fails to extract the correct face regions, via adding small Gaussian
noise to its deepfake images. Second, we find that a face classifier trained
using deepfakes generated by one method cannot detect deepfakes generated by
another method, i.e., an attacker can evade detection via generating deepfakes
using a new method. Third, we find that an attacker can leverage backdoor
attacks developed by the adversarial machine learning community to evade a face
classifier. Our results highlight that deepfake detection should consider the
adversarial nature of the problem.
