{'fraction_non_alphanumeric': 0.027466937945066123, 'fraction_numerical': 0.009155645981688708, 'mean_word_length': 5.0699588477366255, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Functional grammar and the computer

Linguistic theories can be divided into two varieties: those based on the form of linguistic expressions and those based on the function of linguistic expressions. Most presentday theories belong to the former variety. Because of their formal explicitness, the rule systems of these theories can be built into computer applications fairly straightforwardly, although they provide no indication of the contexts and ways in which they should be applied appropriately. Theories based on linguistic function address matters of appropriate usage but frequently fail to supply enough low-level formal detail to facilitate computer implementation. Halliday's Systemic (Functional) Grammar (Butler 1985) is an example of a functional theory that has served as the basis for some interesting computer systems, especially those concerned with text generation. This is probably less attributable to the functional credentials of Systemic Grammar than to its exceptionally useful descriptive tool, the system network (e.g., Mellish 1988). Connolly and Dik's volume presents the first major survey of NLP work based on another theory on functional principles, Dik's Functional Grammar (FG) (Dik 1978). FG views words as predicates. The lexicon consists of a list of predicate frames. A linguistic expression, i.e., a string of predicates, is analyzed in terms of an abstract underlying predication. Predicates and predications make no reference whatsoever to the surface forms of expressions. Predications are related to surface orderings by means of expression rules. These constitute the nearest thing FG has to a syntactic component. Functional Grammar and the Computer consists of six sections containing a total of 16 chapters. Section I is a good introduction by Dik to FG-based NLP and the papers in the collection. He locates the work reported in the book within the framework of a research program bearing the improbable name of FG*C*M*NLU (Functional Grammar Computational Model of the Natural Language User). I have no idea how to pronounce this. The book does not make clear whether FG*C*M*NLU is a single research project with coherent aims or simply the name by which all computational work is known within the FG community. A number of references suggest the former reading (e.g., Kahrel, p. 145; Connolly, p. 218) and this is made all the more plausible by the fact that 12 of the 15 authors are based in Amsterdam. However, many of the papers make no reference to FG*C*M*NLU and instead describe work carried out under rubrics such as ASCOT, LEXALYSE, LIKE, and LINKS. Section 2 ("On Generating'9 consists of two papers devoted to sentence generation by Samuelsdorff and Bakker. Samuelsdorff presents program examples written in PROLOG2, whereas all of the other authors who provide program examples use standard Edinburgh PROLOG. Sentence generation is achieved by applying expression rules to predications.