Extracting Targeted Training Data from ASR Models, and How to Mitigate
  It
  Recent work has designed methods to demonstrate that model updates in ASR
training can leak potentially sensitive attributes of the utterances used in
computing the updates. In this work, we design the first method to demonstrate
information leakage about training data from trained ASR models. We design
Noise Masking, a fill-in-the-blank style method for extracting targeted parts
of training data from trained ASR models. We demonstrate the success of Noise
Masking by using it in four settings for extracting names from the LibriSpeech
dataset used for training a SOTA Conformer model. In particular, we show that
we are able to extract the correct names from masked training utterances with
11.8% accuracy, while the model outputs some name from the train set 55.2% of
the time. Further, we show that even in a setting that uses synthetic audio and
partial transcripts from the test set, our method achieves 2.5% correct name
accuracy (47.7% any name success rate). Lastly, we design Word Dropout, a data
augmentation method that we show when used in training along with MTR, provides
comparable utility as the baseline, along with significantly mitigating
extraction via Noise Masking across the four evaluated settings.
