{'fraction_non_alphanumeric': 0.011933174224343675, 'fraction_numerical': 0.0, 'mean_word_length': 6.424778761061947, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Discovering Linguistic Dependencies with Graphical Models

Graphical models provide a compact approach to analysing and modeling the interaction between attributes. By exploiting marginal and conditional independence relations, high-dimensional distributions are factorized into a set of distributions over lower dimensional subdomains, allowing for a compact representation and efficient reasoning. In this paper, we motivate the choice of linguistic parameters from different language resources as attributes and investigate their interaction. Following that, we extract linguistic dependencies from the structural component of Bayesian Networks induced from data characterized by those attributes. We discuss these preliminary results with respect to their applicability for natural language processing and information retrieval tasks.