{'fraction_non_alphanumeric': 0.05849408836341008, 'fraction_numerical': 0.009956440572495333, 'mean_word_length': 3.757396449704142, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Problems of Malayalam Encoding in the Indic Context

1. In all Indic languages, there are 5 consonant classes (vargas) velar, palatal, retroflex, dental and bilabial. But, in the case of Dravidian languages, there is an additional alveolar class. This class is used with high frequency in Tamil and Malayalam. It is used in a large set of words of the core vocabulary. 2. Another feature to be noted in this context is that in Indo-Aryan languages each class (varga) has 5 instances, whereas in Dravidian languages, there are only 2 (stop and nasal). 3. Unlike Indo-Aryan languages which have many more possible combinations, both within the class (varga) and also between classes, in Dravidian languages, the number of combinations are strictly limited. Each class produces only 3 combinations: 2 geminations and 1 conjunct with the nasal as first member. Also, inter-varga combinations do not exist1. For e.g., in the velar class the three conjuncts are 1. gemination: ക /kka/, ങ /ṅṅa/ 2. combination: ങ /ṅka/ 4. As shown in chart 3, row 6, like all other classes, the alveolar class has the same feature and behavior, i.e., basically it has a stop and a nasal, and combinations are റ /ṯṯa/, ന /ṉṉa/ and ന /ṉṯa/. 5. In Malayalam റ /ṯa/ and ന /ṉa/ do not have their own glyph. They are attained using the glyphs of റ and ന. This is due to some historical reasons (also, grammatically justifiable). In Tamil, ന na has a seperate character from ന /ṉa/, and റ /ṯa/ is attained using the glyph of റ /ṟa/. Thus, the usage of റ for /ṯa/ came from Tamil directly. In the case of ന /ṉa/, in Malayalam, the glyph of