{'fraction_non_alphanumeric': 0.022222222222222223, 'fraction_numerical': 0.00202020202020202, 'mean_word_length': 5.079754601226994, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
On statistical machine translation method for lexicon refinement in speech recognition

In low resource Automatic Speech Recognition (ASR), one usually resorts to the Statistical Machine Translation (SMT) technique to learn transform rules to refine grapheme lexicon. To do this, we face two challenges. One is to generate grapheme sequences from the training data as the targets, which is paired with the original transcripts to train SMT models; the other is to effectively prune the learned rules from the translation model. In this paper we further this study. First we propose a simple but effective pruning method; second, to see in which case we are able to learn better rules, different setups with various acoustic and language model combinations are investigated; finally, to examine if the rules in different setups are complementary, lexicons generated via different rule tables are merged in ASR experiments. We report a WER reduction of up to 6.2% with the proposed technique.