{'fraction_non_alphanumeric': 0.022988505747126436, 'fraction_numerical': 0.000676132521974307, 'mean_word_length': 5.3247863247863245, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Accelerating Rule-Based Reasoning in Disk-Resident RDF Knowledge Bases

Collections of tens of millions of automatically extracted facts represented using the subject-predicate-object RDF model are available for several domains. As big as these collections are, they are unable to capture all information about a domain, simply because the sources from which they were extracted are incomplete. This can be tackled by creating knowledge bases where facts are enforced with rules showing how new facts can be generated from existing ones and constraints which must hold in the relevant domain. Querying such knowledge bases is expensive for two main reasons. First, data is disk resident, which makes access to it slow. Secondly, rule definitions can be recursive, which requires special query evaluation techniques and renders traditional cost-based query optimization and join-ordering techniques less effective. This thesis presents the implementation of a query processor for such a setting. We show how we integrated the RDF-3X RDF query engine into our query processor. We also present optimizations along several dimensions: (i) query evaluation techniques, (ii) caching to reduce both disk access and rule evaluation, (iii) a classification of predicates which allows better utilization of the underlying storage engine’s ability to optimize traditional relational queries and (iv) a novel probabilistic way of looking at join ordering and cost estimation in this context.