{'fraction_non_alphanumeric': 0.022172949002217297, 'fraction_numerical': 0.028824833702882482, 'mean_word_length': 4.512195121951219, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A dop model for phrase structure trees

This chapter gives an instantiation of DOP (Scha 1990, 1992; Bod 1992) which is known as Tree-DOP or DOP1 and which will be extensively used in the rest of this book. Tree-DOP combines subtrees from a treebank to parse new sentences. It employs the relative frequency estimator to assign probabilities to subtrees, and computes the probability of a parse tree as the sum of the probabilities of its derivations.