{'fraction_non_alphanumeric': 0.010460251046025104, 'fraction_numerical': 0.0, 'mean_word_length': 5.6923076923076925, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Token Passing : a Simple Conceptual Model for ConnectedSpeech Recognition

This paper describes a simple but powerful abstract model in which connected word recognition is viewed as a process of passing tokens around a transition network. The advantages of this unifying view are many. The various apparently diierent connected word algorithms can be represented within the same conceptual framework simply by changing the network topology, the application of grammatical constraints is straightforward, and perhaps most importantly, the entire structure is independent of the actual underlying pattern matching technology. To illustrate the power of this conceptual model, the paper concludes by describing some work done under the UK Alvey-sponsored VODIS Project in which the Token Passing paradigm enabled the One Pass algorithm to be straightforwardly extended to include the generation of multiple alternatives and context free syntactic constraints.