{'fraction_non_alphanumeric': 0.033406352683461114, 'fraction_numerical': 0.0, 'mean_word_length': 5.4787234042553195, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Dependency-length minimization and its limits: A possible role for a probabilistic version of the final-over-final condition

Abstract:A prominent principle in explaining a range of word-order regularities is dependency locality, which minimizes the linear distances (dependency lengths) between a head and its dependents. However, it remains unclear to what extent language users in fact observe locality when producing sentences under diverse conditions of cross-categorical harmony (such as the placement of verbal and nominal heads on the same vs. different sides of their dependents), dependency direction (head-final vs. head-initial), and parallel vs. hierarchical dependency structures (e.g. multiple adjectives dependent on the same head vs. nested genitive dependents). Using forty-five dependency-annotated corpora of diverse languages, we find that after controlling for harmony and conditioning on dependency types, dependency-length minimization (DLM) is inversely correlated with the overall presence of head-final dependencies. This anti-DLM effect in sentences with more head-final dependencies is specifically associated with an accumulation of dependents in parallel structures and with disharmonic orders in hierarchical structures. We propose a detailed interpretation of these results and tentatively suggest a role for a probabilistic principle that favors embedding head-initial (e.g. VO) structures inside equally head-initial and thereby length-minimizing structures (e.g. relative clauses after the head noun), while head-final (OV) structures have a less pronounced preference for harmony and DLM. This is in line with earlier findings in research on the Greenbergian word-order universals and with a probabilistic version of what has been suggested more recently as the final-over-final condition.