{'fraction_non_alphanumeric': 0.03758878622163555, 'fraction_numerical': 0.015148657992331385, 'mean_word_length': 4.731268011527377, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Linguistic Knowledge-driven Approach to Chinese Comparative Elements Extraction

The BI ( 比 )-structure, which highlights a contrasting characteristic between two items, is the key comparative sentence structure in Chinese. In this paper, we explore the methods of extracting the 6 constituents of the BI-structure. Previous studies are often restricted to probabilistic classification methods, where the feature used hardly embodies linguistic knowledge, therefore unintuitive. As an alternative, we propose the use of two linguistic knowledge-driven approaches, namely the POS chunking-based and TBL-based methods. The first model effectively captures grammatical restrictions over POS sequential patterns. The second model set up on new and lesser templates performs better than Brill’s (1995). Experimental results show that the proposed models are simple and effective methods for Chinese comparative element extraction task.


Introduction
Comparison is the most representative figure of evaluation. Much of evaluative information is now available in the web, and comparative sentences prevail in Chinese web texts in increasing numbers. A significant amount of research has been conducted on automatic identification of Chinese comparative sentence and its semantic elements. However, the techniques proposed in earlier works are mostly based on statistical classification method. Due to the opaque nature of stochastic features, it is often difficult to comprehend what linguistic aspects are applied to the model.
In this paper, we present a detailed analysis on linguistic behavior of the BI (比)-structure, which is the key comparative structure in Chinese. The application of the two rule-based approaches suggested in this paper are different from previous models in that they fully use syn-tactic and lexical features which are intrinsic to the structure. This paper first presents a brief literature review on the subject. The target of extraction task, i.e. Comparative Elements (CE) is then defined before demonstration of the two proposed approaches, i.e. POS chunking-based and TBLbased extraction models. Finally, we discuss the experiment's results and present our conclusion.

Related Work
The research on comparative sentence has been a main concern from the beginning of modern Chinese linguistics research. The different types of Chinese comparative sentences were first mentioned in Mashi Wentong (1898) and their classification was elaborated later by Chinese grammarians such as Lü (1942), Ding (1961) and Liu (1983). Following by their preliminary work, a series of research focused on defining syntactic and semantic structure of the Chinese comparative sentence was conducted. Li (1986) demonstrates the Chinese BI-structure simplifying rules. Shao (1990) investigates the rule of replacing and omitting elements in Chinese comparative structure.
On the other hand, Studies in Natural Language Processing mainly dealt with the identification of comparative sentence and its elements. Based on Jindal and Liu's research (2006) on comparative sentences in English, Huang(2008) and Song(2009) made a stochastic classifier based on SVMs and CRFs to tackle the Chinese comparative sentence identification and element extraction task. Besides, many models were also suggested in the fifth Chinese Opinion Analysis Evaluation (COAE2013) track. Zhou(2014) and Li(2013) made use of pattern matching technique, and Wei(2013) proposed a rule-based decision making approach based on CRF sequential tag-ging. Despite all these models, their performance has not shown satisfying results 1 . The identification of Chinese comparative elements especially still remains as a big challenge. A TBL-based approach, which showed good performance in Korean (Yang and Ko, 2011), would be an alternative to usual methods.

Comparative Elements (CE)
We refer to Comparative Elements (CE) as entities and attributes which directly occur within comparative sentences. We defined 6 CEs as below.
e.g. 新飞度车身结构的刚度比前代提高了 164％。 The solidity of XinFeiDu's body structure has increased 164% than the previous design. Relative difference in degree or quantity between entities in terms of DIM. Our task is to automatically extract these 6 CEs from the sentences. Note that these elements cannot simply be determined by syntactic criteria. They are involved with semantic category to some extent, but we do not use additional semantic features such as semantic role labels or lexical taxonomies in this paper.

Corpus
The corpus used in this experiment consists of 1,036 Chinese BI-structure sentences, coming from the open dataset of COAE 2013 Task 2 (Tan et al. 2013). The sentences are a collection of customer reviews and opinions from different Chinese websites pertaining to cars and electronics. 1) Preprocessing: We first conduct word segmentation and POS tagging by using ICTCLAS 3 . Second, we had to manually revise to avoid any errors because of the informal language used on the web. Three annotators were appointed to revise typos. In addition, 3,000 word-size domain-specific lexicons 4 are also utilized to guarantee the quality of word segmentation and POS tagging.
2) CE labeling: The 6 types of Comparative Elements (CE) in the 1,036 sentences were manually annotated with the corresponding CE labels of Table 1. This task was done by three trained annotators of Chinese linguistics major. Their work was double-checked by one another, and any inconsistencies between annotators were discussed before reaching an agreement. The annotated corpus was then transformed to IOB format.

Two methods of Comparative Elements Extraction
We now present two different proposed techniques. Model 1 uses basic part-of-speech chunking-based method and Model 2 employs Transformation-Based Error-Driven Learning (TBL) (Brill, 1995) for identifying CEs. Punctuation as delimiter, the sentence is divided into small clauses If the clause contains"比/p", the following chunk rules are applied to create chunks. For RES chunks, Rule 5 is applied to chunk EXT.  We now give a step-by-step illustration of the actual extraction process of Table 2.

POS chunking-based CE extraction
Step 1 Detecting BI (比)-Chunk For clauses containing the comparative marker BI (比), we create a chunk that begins with it (Rule 1). We call it BI (比)-Chunk.
Step 2 Splitting into Two Phrases BI (比)-Chunk can be divided into two chunks, i.e. BI-prepositional phrase and predicate because they belong to very distinctive syntactic categories. The former cannot appear independently, usually taking a noun as its object. The latter functions as the predicate, and is mostly an adjective or a verb 6 . Therefore, we designed Rule 2 to split them.
Step 3 Merging Incorrectly Separated Predicates In most cases, however, the predicate is a complex phrasal structure. Therefore, Rule 2 incorrectly splits chunk that should have not been sep-5 For specifying chunk rules intuitively, we directly quote NLTK's description of chunking operator (Bird et al. 2009).
(1) <T> represents for any token tagged with T.
(2) {<pattern >} represents for Chunk Rule, which means creating a chunk with the given regex pattern within curly braces.
(3) <pattern1>}{<pattern2> represents for Split Rule, which means splitting a chunk into two chunks based on the specified pattern. (4) <pattern1>{}<pattern2> represents for Merge Rule, which means merging two chunks together based on the specified pattern. 6 Strictly speaking, verb and adjective are also able to occur in BI-prepositional phrase. Such a case will be handled in Step 4. arated. To solve this, we employ Rule 3 to merge incorrectly divided elements of the predicate group.
Step 4 Dealing with DE (的)-Structure In Chinese, DE (的) is often used to mark modification 7 . It can be attached to various types of syntactic categories and modify the following word. DE ( 的 )-structure can be simplified as [ [ XP 的 ] NP ]. When a verb or adjective phrase takes the position of XP, the same error as in Step 3 occurs. To tackle this problem, we use Rule 4 that enunciates the unity of modifying elements occurring at the position of XP.
Note that DE (的) is not necessarily restricted to modification marker. When occurring at the end of the sentence, it simply marks a subjective tone. Rule 4 makes use of punctuation tag (WJ) to discern this modal particle of DE (的) from modification marker.
Step 5 CE Labeling After successfully extracting the two chunks (BIprepositional phrase and predicate) following the above mentioned 4 steps, we label each item in these chunks with BI, OBJ and RES tags.
Step 6 (optional) EXT Identification Comparative Extent (EXT) usually begins with numerals, following the head of the predicate. Rule 5 detects possible EXTs in RES chunk.

TBL-based CE extraction
The advantage of using the POS chunking-based method is that it allows direct capture of linguistic information. However, (a) it requires painstaking process of manual rule construction; (b) and an error in any step could damage the performance of the whole CE extraction process.

Transformation-Based Learning
We tested an automated learning method, known as Transformation-Based Error-Driven Learning (TBL) (Brill, 1995). The basic idea of TBL is "learning from mistakes". First, the researcher may apply an initial-state annotator to the training corpus. Second, the set of user-defined templates are then used to form candidate rules. Third, each of the rules is in turn applied to the training corpus. At the same time, the net improvement of the rule is calculated and recorded for evaluation of candidate rules. Throughout the training, the process of deriving rules, scoring and selecting rules and applying them is iterated, creating an ordered sequence of transformation rules.  Brill(1995), Ramshaw and Marcus(1999) TBL has a well-known advantage, i.e. perspicuity of linguistically meaningful rules. Different from the POS chunking-based model, the TBLbased model is more robust and possibly captures useful information that may not be noticed by the human engineer (Brill, 1995:552). Therefore, the use of TBL allows us to capture some otherwise ignored CE patterns.

Preliminary TBL-based CE Tagger
We treated CE extraction task as a tagging problem. Each token in training data is given an initial tag by ICTCLAS tagger. The TBL learner with user-defined templates is then trained on the training data. Consequently, we obtain the TBLbased CE tagger as a CE extraction model.
The templates play a key role in this model because the TBL-learner uses these templates to generate possible rules, which directly affect the overall performance. In order to see which type of feature contributes more to the TBL-based model accuracy rate, we divide the original template proposed in Brill (1995:553, 556)   Since the result of (c) is the best, we take it as a standard template. Table 4 shows evaluation results using TBL-based model with 26 templates (c). We regard this score as our baseline performance.

Search for Optimal Template
We now present how we obtained our proposed TBL-based CE extraction model. The baseline system is based on a relatively large amount of rules and templates. Therefore, the reduction of rules is preferable for efficient application of TBL. According to Brill (1995: 560), although the accuracy of TBL-based tagger increases with the number of transformation rules, its marginal effect dramatically decreases, and leads to computational cost. We found that 200~300 rules are desirable for our CE detecting task. As for templates, we achieved the best performance when using tag and lexicon sequences within a radius of 3 tokens as features.
For every token in BI-structure, change tag a to tag b when: 1. The current word is w. W 0 2. One of the three preceding words is tagged z.
T 1, 2, 3 4. The word two after is tagged z. T 2 5. The word two before is tagged z.
T -2 6. The following word is tagged z.
T 1 7. The preceding word is tagged z.
T -1 8. The preceding word is w.
W -1 9. The current word is w, and the preceding tag is t.
10. The preceding tag is t, the current tag is t 2 and the following word is w.

Result of POS Chunking-based Model
The overall performance of the chunking-based CE extraction model (Section 4.1) is as follow.  The CE mining process of chunking-based model is based on simplistic grammatical assumptions: (a) Only nominal elements serve as the complement of BI-prepositional phrase; (b) Predicates can be a word or a group of words (phrase) that are adjectives or verbs; (c) Within the BI (比)-Chunk, the elements occurring before the modifier marker DE (的) are all regarded as modifier. These assumptions, of course, are somewhat over-generalized, and do not fit in many real cases 8 . However, the 5 Rules applied based on these assumptions show a fair performance in Table 7 when applied to a limited scope of BI-Chunk.

Result of TBL-based Model
All evaluations of TBL-based model in this paper are based on a 5-fold cross validation. The proposed TBL-based model with 10 templates shows the results below.  Guided by our new templates (Table 5), the model first locates comparative marker BI (比), then searches the surroundings for the tag/lexical features while gradually narrowing its scope. As a result, the 10 templates enable an effective detection of elusive CE instances such as those in example (2-7).  Table 9: Comparison between models (f-score, %) Table 9 compares the scores of two CE mining methods, the POS chunking-based and the TBLbased approach. Compared to the TBL-based model, the POS chunking-based model is unable to extract SUB and DIM. Because these two elements frequently occur outside a BIprepositional phrase, it is hard to capture their irregular occurrence positions in the sentence. In contrast, the TBL-based model is able to detect SUB and DIM. However, their identification rate is relatively low. Nevertheless, our proposed TBL-based model outperforms the baseline system by using much smaller templates. It shows we found a simple and more expressive set of rule templates.
Moreover, the proposed TBL-based model achieved an increase of 21% for RES and 14% for EXT f-score in comparison with the POS chunking-based model. This improvement mainly benefits from the proper use of both tag and lexical information.
(6) 市场/nz 中/f 比/p 它/rr 靓/a 的/ude1 产品/n 很/d 少/a 。/wj There are very few products prettier than that one in the market. As for examples (6-7), the POS chunking-based model (Section 4.1) incorrectly identifies "少, 近" as RES. As we can see in example (6a), the POS chunking-based model wrongly identifies "很少" as RES because the BI-chunk "比它靓" occurs in front of the modification marker "的". In (7a), the model mistook "近" for RES because it cannot discern "近" from "贵" only with the tag information. In contrast, TBL-based model makes a correct decision of (6b) and (7b) based on lexical information.

Conclusion
In order to make the best use of meaningful features in linguistic context, we have proposed the use of two rule-based methods for Chinese comparative element (CE) extraction. The POS chunking-based model performs well with basic Chinese grammatical rules. We then use the TBL-based method to extract other linguistic patterns that the first model can hardly detect. Results showed that our TBL-based model achieved higher score than Brill's (1995), demonstrating that our new 10 templates can effectively extract the distinct features of Chinese BI-structure as shown in examples (1-7). Chinese comparative element mining involves techniques of various domains including coreference resolution, named entity recognition and parsing. However, the linguistic features used in this paper are limited to instances of regular (type-3) grammars. In our future work, we plan to investigate some feasible Chinese linguistic features on the level of context-free grammars.