{'fraction_non_alphanumeric': 0.023504956827630315, 'fraction_numerical': 0.008234729772945315, 'mean_word_length': 4.988032551460028, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adaptive Word Sense Tagging on Chinese Corpus

This study describes a general framework for adaptive word sense disambiguation. The proposed framework begins with knowledge acquisition from the relatively easy context of a corpus. The proposed framework heavily relies on the adaptive step that enriches the initial knowledge base with knowledge gleaned from the partially disambiguated text. Once adjusted to fit the text at hand, the knowledge base is applied to the text again to finalize the disambiguation decision. The effectiveness of this approach was examined through sentences from the Sinica corpus. Experimental results indicated that adaptation significantly improved the performance of WSD. Moreover, the adaptive approach, achieved an applicability improvement from 33.0% up to 74.9% with a comparable precision.


Introduction
Word sense disambiguation is a long-standing problem in natural language understanding. Statistically acquiring sufficient knowledge about a language to build a robust WSD system is extremely difficult. For such a system to be efficient, a large mass of balanced materials must be gathered to cover many idiosyncratic facets of the language. Three issues must be addressed in a lexicalized statistical word sense disambiguation (WSD) model: data sparseness, lack of abstraction, and static learning. First, a word-based model has a multiplicity of parameters that are difficult to measure consistently, even with an extremely large corpus. Under-trained models lead to low precision. Second, word-based models lack a crucial degree of abstraction for a broad coverage system. Third, a static WSD model is probably neither robust nor portable, since it is difficult to construct a model relevant to a broad range of unrestricted texts. Several WSD systems have been created that apply word-based models to a specific domain to disambiguate senses appearing in generally easy contexts with a large number of typically salient words. In an unrestricted text, however, the context is usually diverse and difficult to capture with a lexicalized model; therefore, a corpus-trained system is unlikely to transfer suitably to a new domain. Generality and adaptability are, therefore, essential to a robust and portable WSD system. An adaptive system, armed with an initial knowledge base extracted from defined words, is superior in two ways to static word-based models trained on a corpus. First, the initial knowledge is sufficiently rich and unbiased for a large portion of text to be disambiguated correctly. Second, based on the initial disambiguation, an adaptation step can then be implemented render the knowledge base more relevant to the task, thus resulting in broader and more precise WSD.
This study explores in detail whether word-based knowledge provides a general solution for disambiguating contexts of unrestricted texts. This method assumes that a major part of a given text is easy or prototypical and, therefore, understandable using general knowledge. Adapting contextual representation of word senses to those in the easy context, will hopefully allow us to interpret the other part, which is normally considered a hard context. Adaptation makes the knowledge base more relevant to the text and, therefore, more effective for WSD in a hard context. Experimental results demonstrate the feasibility of this adaptive WSD approach.
The rest of this paper is organized as follows. Section 2 reviews recent WSD literature from the viewpoints of various contextual knowledge types and different representation systems. Section 3 then describes the strategy of using the adapted knowledge base and default is described. Next, Section 4 provides a detailed account of experiments conducted to evaluate the effectiveness of the adaptive approach, including the experiment setup, results and evaluation. Conclusions are finally made in Section 5.

Previous Works
Using a machine to select the chosen sense of a polysemous word in a specific context has received increasing interest. Various methods of WSD have been proposed recently in natural language processing literature, and newer ones have rapidly superseded old ideas. Central to these efforts are the contextual knowledge encoded and the way this knowledge is described. This section reviews recent WSD literature from the viewpoints of different forms of contextual knowledge and their representational schemes.
Any scheme for gaining contextual information on word sense must begin with means of identifying the word sense, since word sense is an abstract concept, unclear on the surface. With this completed, the surrounding words to construct a contextual representation of the word sense for WSD. Three approaches are available to divide word senses. First, human means can be used to derive a hand-tagged corpus of word senses. Earlier WSD works adopted this approach and hand tagged the intended sense of each polysemous word in the training corpus (Kelly and Stone 1975;Hearst 1991). Second, the numbered sense entries readily available in a machine-readable dictionary can be taken, with their definitions and examples treated as contextual information (Lesk 1986;Veronis and Ide 1990;Wilks et al. 1990;Guthrie et al. 1991). The third way of eliciting word sense uses linguistic constraints. For instance, three linguistic constraints can be exploited for successful sense tagging and WSD.
One sense per discourse The senses of all instances of a polysemous word are highly consistent within any given document.
One sense per collocation Words in close proximity offer strong and consistent clues to the sense of a target word, conditional on the relative distance, order, and syntactic relationship.
One sense per translation Translations in a bilingual corpus can represent the senses of words.
To exemplify the first constraint, consider the word suit. The constraint captures the intuition that if the first occurrence of suit is a LAWSUIT sense, then later occurrences in the same discourse are also likely to refer to LAWSUIT (Gale, Church and Yarowsky 1992a). The second constraint reveals that most works on statistical disambiguation have assumed that word sense is closely correlated with particular contextual features, like occurrence of particular words in a window around the ambiguous word. However, Yarowsky (1995) proposed that strong collocations should be identified for WSD. In a bilingual corpus, differences in translations of the polysemous word allowed one to identify the intended sense, particularly in contrasting polysemy. Gale, Church and Yarowsky (1992b) used French translations in parallel texts to disambiguate some polysemous words in English. For instance, the senses of duty were typically translated as two different French words, droit and devoir, respectively, representing the senses tax and obligation. Thus, a number of tax sense examples of duty could be collected by extracting instances of duty that were translated as droit, and the same could be done for obligation sense examples of duty.
Once word senses are identified, the context of a particular word sense can then be gathered and encoded in some way for use in the following disambiguation step. At least two ways are available to encode contextual knowledge. The obvious way, the lexicalized representation, is a surface scheme that keeps a weighted list of words occurring in the context of a particular sense. Conversely, the conceptual representation encodes the categories of words that might appear in the context.

An Adaptive Method for WSD
The first step of this study was to construct an initial knowledge from training corpus, then to describe how the knowledge was employed to resolve ambiguity for polysemous words in a context.

Construct an Initial Knowledge from Training Set
To avoid accumulating extraneous information in the knowledge acquisition, sense information was only adopt from the easy text in context during the acquisition phase. First, a preparatory segmentation was made for the training material. Next, target words in each sentence were labelled associated senses. A knowledge base was then construct based on the occurrence frequency of the surrounding target words in each sentence. Finally, the target words' co-occurrence probability was computed and the above descriptive outline of the procedure was summed up as Algorithm 1.
Algorithm 1: Step 1: Let D w denote a set of sentence collections consisting of a target word w in training material.
Step 2: For each sense division s of word w, let count(c, w, s) denote the frequency count that word c occurs in all word occurrences for word w in sense s.
Step 3: For each target word w, and its sense division s, the co-occurrence probability Pr(c, w, s) of possible context word c and w is calculated as the following:

Adaptive Sense Disambiguation
This section took advantage of an adaptive approach to automatically resolve ambiguity for polysemous words in context. Naïve Bayes is a simple but effective text classification algorithm for learning from labelled data alone (Lewis, 1998;McCallum and Nigam, 1998). The parameterization given by Naïve Bayes defines an underlying generative model assumed by the classifier. Considering the sense-tagging task as a classification problem this model assumes each word in a sentence was generated separately from the others, given the word sense.
The adaptive process began with an initial collection of labelled sentences L and one of unlabeled sentences set U. For all instances of polysemous word w in U and each sense s' of w, the sense-conditional probabilities score(s'| w, S) was computed. Next, the ratio R(w, S) was computed for all instances of w in U, and the most secure instance S was picked from U. Following that, the instance was labelled to sense s 1 and S added to L. Next, the same process was performed on sentences in U until they had been disambiguated. Algorithm 2 which gives a formal and detailed description of adaptive WSD, is shown as follows: Algorithm 2: Step 1: If un-labeled test set U is an empty set, stop.
Step 2: For all instances of polysemous word w in U and each sense s' of w, compute the sense-conditional probabilities score(s'| w, S).
Step 3: Compute R(w, S) for all instances of polysemous word w in U. Step 4: Pick the most secure instance S with the largest value of R(w, S) and R(w, S) is greater than a preset threshold, θ. Then, label sense s 1 to S and add it to L.

Experiment
Two experiments were conducted to assess the effectiveness of the proposed method: a WSD experiment with adaptive process and an experiment without adaptive process. The experimental setup is described in a number of steps as follows.
(1) A set of 20 polysemous words was chosen as the target for disambiguation and evaluation. Table 1 lists these words. The senses number from 2 to 8 of these words and their average sense numbers are 3.2.
(2) For each polysemous word, a sense division was established based on the Chinese WordNet (Miller, 1990;Fellbaum, 1998;CKIP, 2003). (3) Tests were performed on the sentences from the Sinica corpus (CKIP, 1995). The ambiguity of these testing words in our experiment is shown as Table 1.

Evaluation
To assess the performance, two human judges were asked to give a sense label to each example of these twenty words in the testing set. The results of running the two programs on the testing set were compared against those of human assessors. The number of test instances and correct assignments in these two experiments were tallied to produce the precision rate for each experiment. Tables 2 and Table 3are summarized the experimental results. Based on the results, the adapting approach was reasonably helpful for WSD, achieving an applicability improvement from 33.0% up to 74.9% with comparable precision. Table 4 described the experimental precision and applicability for each run.

Conclusion
This study presented an adaptive approach to word sense disambiguation. Under this novel learning strategy, an initial knowledge set for WSD was first built based on the sense definition in training data. These disambiguated texts can be used to adjust the fundamental knowledge in an adaptive fashion so to improve disambiguation precision. We have demonstrated that this approach can outperform established static approaches based on direct comparison of experimental results. This level of performance is achieved without lengthy training or the use of a very large training corpus.