{'fraction_non_alphanumeric': 0.030837004405286344, 'fraction_numerical': 0.0, 'mean_word_length': 4.875, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
On the Utility of Recovering Dropped Words in Chinese Named Entity Recognition

Named entity recognition (NER) in natural language processing (NLP) considers the problem of identifying a sequence of words in a sentence text that mentions a predefined type of object (entity), e.g., person, organization, location, or time. NER methods are keys in extracting knowledge from texts as entities are fundamental for attaching entity properties or entity relations. However, NER for texts in Chinese is trickier due to that some auxiliary words maybe dropped in a sentence, which is a common phenomenon in Chinese writing for brevity. A usually dropped Chinese word is ‘的’ (often functions as the word ‘of’ in English). One obvious effect of this kind of omitting is bring difficulty in identifying the sub-entities (or nested named entities) contained in a named entity. Previous works considers the effected of recovering dropped pronouns in the Chinese translation task. Here we proposed a rule-based method to rover the auxiliary word ‘的’ for Chinese text, and study the effect of this recovery on the performance of a state-of-the-art Chinese NER method FLAT. Experimental results on Weibo-NER and MSRA-NER datasets shows that our method improves on FLAT. This study thus highlights the promising of recovering more types of dropped words for Chinese NER problem.