{'fraction_non_alphanumeric': 0.014184397163120567, 'fraction_numerical': 0.0, 'mean_word_length': 4.947368421052632, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Word Similarity Calculation Method Based on CSD and Statistics

The calculating of similarity is an important process in NLP. As the basis of sentence and article similarity, the research on calculating of word similarity seems especially vital. There are two word similarity calculating methods: semantic based and statistics based. Former semantic based method mostly relies on Wordnet or Hownet. This article introduces a new method which is based on CSD and which also addes a statistics based method. It offers good support for further processing of corpus.