{'fraction_non_alphanumeric': 0.028192371475953566, 'fraction_numerical': 0.006633499170812604, 'mean_word_length': 5.22680412371134, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Translation Framework for Visually Grounded Spoken Unit Discovery

Multimodal acoustic unit discovery (MAUD) is a key task in self-supervised spoken language learning and low-resource speech recognition. In this paper, we proposed two models for MAUD inspired by machine translation models where we treat speech and image as source and target languages. Our word discovery model outperforms previous state-of-the-art approach by 5.3% alignment F1 on SpeechCOCO dataset and our phoneme discovery model outperforms previous state-of-the-art approach by 7% normalized mutual information on TIMIT dataset.