{'fraction_non_alphanumeric': 0.021379980563654033, 'fraction_numerical': 0.009718172983479106, 'mean_word_length': 5.560509554140127, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Properties of consonant sequences within words and across word boundaries

This paper is concerned with the problem of locating word boundaries from phonemic strings. As a step towards a better understanding of the acoustic phonetic properties of consonant sequences within and across word boundaries, we studied the distributional properties of these sequences. The database consists of several text files from different discourses, ranging in size from 200 to 38,000 words. In each case we tabulated the number of distinct word-initial, word-medial, word-final, and word-boundary consonant sequences and their frequency of occurrence. Word-medial consonant sequences represent a small subset of those found across word boundaries. Of the non-medial consonant sequences, approximately 80% have a unique boundary location. When the consonant sequence cannot uniquely specify the location of a word boundary, different placement of word boundaries often results in allophones with substantially different acoustic characteristics.