{'fraction_non_alphanumeric': 0.02091439688715953, 'fraction_numerical': 0.0019455252918287938, 'mean_word_length': 4.761904761904762, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Analogy in Word Formation

One of the salient properties of language is that it has two faces: dynamic and static. We often attribute the uniqueness of human language to the fact that it can generate infinite number of sentences to express infinite number of propositions, but at the same time it has a finite set of words that are combined to form a sentence. Roughly speaking, the former property can be relegated to syntax and the latter to the lexicon, so syntax represents the dynamic aspect of language, while in the lexicon we find a finite list of words memorized by the speaker of that language. Thus it is often pointed out that while it is possible to list the memorized words as a dictionary, it is utterly impossible to compile all the potential sentences of one language. When we look closely into what constitutes the mental lexicon (our knowledge about the words of one language), however, we see that it is not merely static. There are word formation processes that create words out of existing ones: taking happy as an example, prefixation of underives unhappy, suffixation of –ness happiness, and compounding trigger-happy and happy hour . Given such word formation rules, can we say that the lexicon is as dynamic as syntax is? The answer is clearly no, since many of the outputs of word formation rules are themselves listed in the lexicon because many of such outputs are only potential words and not actual words (Aronoff (1973)): unhappy vs. #unsad . In other words, even though word formation rules can combine words just as syntactic rules do, there is a question of productivity of the process. Some word formation processes are extremely productive and most of the outputs are actual words, while others are not productive (or idiosyncratic) and produce only a handful of existing words. Such varying degrees of productivity are never observed in syntactic rules; even though there may be some constraints to limit the application of syntactic rules, the grammaticality of the output sentence is always predictable by principle.