{'fraction_non_alphanumeric': 0.020689655172413793, 'fraction_numerical': 0.0, 'mean_word_length': 5.275675675675676, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A word graph interface for a flexible concept based speech understanding framework

In this paper, we introduce a word graph interface between speech and natural language processing systems within a ﬂexible speech understanding framework based on stochastic concept modeling augmented with background ”ﬁller” models. Each concept represents a set of phrases ( written as a context free grammar (CFG)) with the same meaning, and is compiled into a stochastic recursive transition network (SRTN). The arcs (or rules) are tagged with probabilities after training. The ﬁller models are used for phrases that are not covered by the concept networks. The structure in concept+ﬁller sequences is captured by n -grams. The interface is implemented within the context of CU Communicator spoken dialog system. We investigate the effect of several different ﬁller models and interpolation of complementary language models on the system performance. We report notable performance improvements compared to the baseline system. The gain in performance along with the efﬁ-ciency and ﬂexibility of the method motivates future work on the implementation of a tighter interface.