{'fraction_non_alphanumeric': 0.033099297893681046, 'fraction_numerical': 0.0010030090270812437, 'mean_word_length': 5.743243243243243, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A three-stage solution for flexible vocabulary speech understanding

This paper discusses our three-stage approach to a flexible vocabulary speech understanding system, which can detect out-ofvocabulary (OOV) words, and hypothesize their phonetic and orthographic transcriptions. In the first stage, we introduce the column-bigram finite-state transducer (FST) which, while embedding ANGIE sublexical models, also supports previously unseen data from unknown words. Secondly, the ANGIE models utilize grapheme information, providing tighter linguistic constraint as well as instantaneous sound-to-letter capability during recognition. Thirdly, the syllable-level lexical units of the first stage are automatically derived via an iterative procedure to optimize performance. The second-stage recognizer employs ANGIE to output a word network which is parsed by TINA, our natural language (NL) processor, in stage three. Experiments with a JUPITER implementation of this system are described in [1].