{'fraction_non_alphanumeric': 0.018010291595197257, 'fraction_numerical': 0.0, 'mean_word_length': 5.274193548387097, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Alignment-Based Method for Learning Coordinate Phrases in Biomedical Abstracts

Kurohashi and Nagao’s coordination detection technique is an effective method for detecting coordinations in Japanese. Because the method involves several heuristic rules designed specifically for Japanese, these rules must be adjusted if it is to be applied to other languages and domains. In this paper, we propose a general perceptron learning model for adjusting the rules automatically to a given text collection. In order to do so, we reformulate Kurohashi and Nagao’s method in terms of sequence alignment so that adjusting the heuristic rules is reduced to the learning of the substitution matrix. Our method also allows training examples in which only the start and end of coordinate phrases are marked, reducing the cost of training example annotation. We applied the method to the detection of coordinate noun phrases in the GENIA corpus, a collection of English biomedical abstracts. Although minimal features (only surface and part-of-speech of words) were employed, we obtained a coordination detection accuracy competitive to a text chunker and Charniak’s full parser.