{'fraction_non_alphanumeric': 0.015063168124392614, 'fraction_numerical': 0.001943634596695821, 'mean_word_length': 5.315950920245399, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Inducing domain theories

Inducing Domain Theories A thesis submitted to the Committee for Linguistics, Philology and Phonetics in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the University of Oxford. by Maria Liakata, St Edmund Hall, Trinity Term 2004 This thesis presents a method for learning a domain theory automatically from a corpus of parsed sentences. What is meant by a ‘domain theory’ is a collection of facts and generalisations or rules which capture what commonly happens (or does not happen) in some domain of interest. As language users we implicitly draw on such theories in various disambiguation tasks, such as anaphora resolution and prepositional phrase attachment, and formal encodings of domain theories can be used for this purpose in natural language processing. Domain theories may also be objects of interest in their own right, that is, as the output of a knowledge discovery process, providing previously unobserved information to aid with the understanding of the domain. The learning paradigm employed is Inductive Logic Programming (ILP), which generalises over examples from the domain to obtain more general patterns covering the majority of the input instances. ILP was preferred over other machine learning techniques due to the expressive power of the language specifications guiding the search for general patterns and the fact that it allows the inclusion of background knowledge. Different ILP algorithms were explored in conjunction with the domain of company succession events. The relational data mining algorithm WARMR gave the most satisfactory results as it was able to capture frequent patterns of complex structure, often encoding causal relations, consisting of two or more verbs and information about their respective arguments. Finite State Automata (FSA) minimisation techniques were employed to render the rules learnt into a more compact format. Thus, sets of rules were represented as weighted FSAs, which are easier for humans to visualise than a first order logic