{'fraction_non_alphanumeric': 0.018984337921214997, 'fraction_numerical': 0.0, 'mean_word_length': 5.2925373134328355, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The critic as expert

In the world of computer applications and literary text analysis, natural language processing began with words, word lists, word indexes, and words in context — what we now call concordances. These concordances were conceived and developed as basic research tools to allow the scholar to base his or her analyses of a particular text on some hard facts. The actual number of occurrences of a word, previously thought only intuitively to be dominant, and the different contexts in which such words appeared were considered essential information for proving hypotheses of one kind or another. Humanists have been using such concordances for years to help them make generalizations about large bodies of texts. After a while, with increased expertise, lemmatization (the classification of words under their dictionary heading) and collocation (the regrouping of strings of lexical items) became the new buzz words. And rightly so, for these new classification procedures allowed for more complex studies, more interesting comparisons, and more types of studies — be they syntactic, thematic, or otherwise. Another innovation was the introduction of more and more sophisticated, quantitative methods of manipulation of natural language data bases. Thanks to this type of calculation, experts in the field now provide us with a variety of numeric frequency and distribution profiles of lexical items — by doing type/token analysis, by calculating zscores, by using the chi-square test, etc. Computer-oriented critics now often produce useful and impressive linear and multidimensional graphs of the distribution and relationship of any number of complex sets of variables. They have also begun to use more or less complex and flexible parsing programs and comparisons with specialized dictionaries to dissect and analyze text. As a quick look through such journals as Computers in the Humanities and Literary and Linguistic Computing will show, the applications of computers to textual analysis are numerous and varied. All of these innovations and applications have proven extremely useful