{'fraction_non_alphanumeric': 0.013869625520110958, 'fraction_numerical': 0.0027739251040221915, 'mean_word_length': 5.446428571428571, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Extensible pattern matching in SNOBOL4

Although the string pattern-matching facilities of SNOBOL4 are the most powerful of those of any widely used programming language, they suffer both from complexity and lack of a mechanism for defining new scanning procedures. This paper describes a definitional mechanism that provides extensibility for the existing facilities in which programmer-defined scanning procedures are written as co-routines at the source-language level. The result provides a substantial increase in power as well as the potential for simplifying the existing language. The relationship of this new facility to the present language is discussed, as are implications for more extensive language changes.