{'fraction_non_alphanumeric': 0.004724409448818898, 'fraction_numerical': 0.0, 'mean_word_length': 5.235294117647059, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Classification of Japanese Documents and Ranking of Representative Documents by Using the Characteristic of the Frequencies of Words

We developed a method for classification of Japanese documents and ranking of representative documents by using the characteristic of the frequencies of nouns. A representative document is defined as a document whose feature vector is the closest to the center of gravity of the class in the feature vector space among all documents belonging to the class belonging to the class. The ranking of representative documents is decided in descending order of the number of documents belonging to the class.