{'fraction_non_alphanumeric': 0.012895662368112544, 'fraction_numerical': 0.007033997655334115, 'mean_word_length': 5.1, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A corpus-based study on random textual vocabulary coverage

Abstract This paper investigates the random textual vocabulary coverage of the nine domains in the BNC using sets of five hundred 2000-word samples randomly drawn from each of the domains. Random textual vocabulary coverage is the coverage by the vocabulary of a text or a collection of texts of any given length over that of another text or a collection of texts of any given length. The estimation of random textual vocabulary coverage depends on the determination of the relationship between vocabulary size and text length. Brunet's model proves to be robust in capturing such relationship. A mathematical estimator for random textual vocabulary coverage is developed incorporating Brunet's model. A method for computing the 95% random textual vocabulary coverage interval is also devised.