Fingerprinting Fine-tuned Language Models in the Wild
  There are concerns that the ability of language models (LMs) to generate high
quality synthetic text can be misused to launch spam, disinformation, or
propaganda. Therefore, the research community is actively working on developing
approaches to detect whether a given text is organic or synthetic. While this
is a useful first step, it is important to be able to further fingerprint the
author LM to attribute its origin. Prior work on fingerprinting LMs is limited
to attributing synthetic text generated by a handful (usually < 10) of
pre-trained LMs. However, LMs such as GPT2 are commonly fine-tuned in a myriad
of ways (e.g., on a domain-specific text corpus) before being used to generate
synthetic text. It is challenging to fingerprinting fine-tuned LMs because the
universe of fine-tuned LMs is much larger in realistic scenarios. To address
this challenge, we study the problem of large-scale fingerprinting of
fine-tuned LMs in the wild. Using a real-world dataset of synthetic text
generated by 108 different fine-tuned LMs, we conduct comprehensive experiments
to demonstrate the limitations of existing fingerprinting approaches. Our
results show that fine-tuning itself is the most effective in attributing the
synthetic text generated by fine-tuned LMs.
