Deduplicating Training Data Mitigates Privacy Risks in Language Models
  Past work has shown that large language models are susceptible to privacy
attacks, where adversaries generate sequences from a trained model and detect
which sequences are memorized from the training set. In this work, we show that
the success of these attacks is largely due to duplication in commonly used
web-scraped training sets. We first show that the rate at which language models
regenerate training sequences is superlinearly related to a sequence's count in
the training set. For instance, a sequence that is present 10 times in the
training data is on average generated ~1000 times more often than a sequence
that is present only once. We next show that existing methods for detecting
memorized sequences have near-chance accuracy on non-duplicated training
sequences. Finally, we find that after applying methods to deduplicate training
data, language models are considerably more secure against these types of
privacy attacks. Taken together, our results motivate an increased focus on
deduplication in privacy-sensitive applications and a reevaluation of the
practicality of existing privacy attacks.
