{'fraction_non_alphanumeric': 0.029705644072373753, 'fraction_numerical': 0.006683769916284094, 'mean_word_length': 4.466051660516605, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Word Sense Disambiguation based on term to term similarity in a context space

This paper describes the exemplar based approach presented by UNED at Senseval-3. Instead of representing contexts as bags of terms and defining a similarity measure between contexts, we propose to represent terms as bags of contexts and define a similarity measure between terms. Thus, words, lemmas and senses are represented in the same space (the context space), and similarity measures can be defined between them. New contexts are transformed into this representation in order to calculate their similarity to the candidate senses. We show how standard similarity measures obtain better results in this framework. A new similarity measure in the context space is proposed for selecting the senses and performing disambiguation. Results of this approach at Senseval-3 are here reported.


Introduction
Word Sense Disambiguation (WSD) is the task of deciding the appropriate sense for a particular use of a polysemous word, given its textual or discursive context. A previous non trivial step is to determine the inventory of meanings potentially attributable to that word. For this reason, WSD in Senseval is reformulated as a classification problem where a dictionary becomes the class inventory. The disambiguation process, then, consists in assigning one or more of these classes to the ambiguous word in the given context. The Senseval evaluation forum provides a controlled framework where different WSD systems can be tested and compared.
Corpus-based methods have offered encouraging results in the last years. This kind of methods profits from statistics on a training corpus, and Machine Learning (ML) algorithms to produce a classifier. Learning algorithms can be divided in two main categories: Supervised (where the correct answer for each piece of training is provided) and Unsupervised (where the training data is given without any answer indication). Tests at Senseval-3 are made in various languages for which two main tasks are proposed: an all-words task and a lexical sample task. Participants have available a training corpus, a set of test examples and a sense inventory in each language. The training corpora are available in a labelled and a unlabelled format; the former is mainly for supervised systems and the latter mainly for the unsupervised ones.
We report here the exemplar-based approach developed by UNED and tested at the Senseval-3 competition in the lexical sample tasks for English, Spanish, Catalan and Italian.
After this brief introduction, Sections 2 and 3 are devoted, respectively, to the training data and the processing performed over these data. Section 4 characterizes the UNED WSD system. First, we describe the general approach based on the representation of words, lemmas and senses in a Context Space. Then, we show how results are improved by applying standard similarity measures as cosine in this Context Space. Once the representation framework is established, we define the criteria underlying the final similarity measure used at Senseval-3, and we compare it with the previous similarity measures. Section 5 reports the official results obtained at the Senseval-3 Lexical Sample tasks for English, Spanish, Italian and Catalan. Finally, we conclude and point out some future work.
surrounding context, where the average context window varies from language to language. Each training example gives one or more semantic labels for the ambiguous word corresponding to the correct sense in that context. Senseval-3 provided the training data and the test data in XML format. The XML tagging conventions provides an excellent ground for the corpora processing, allowing a simple way for the data browsing and transformation. However, some of the XML well-formedness constraints are not completely satisfied. For example, there is no XML declaration and no root element in the English Lexical Sample documents. Once these shortcomings are fixed any XML parser can normally read and process the data.
Despite the similarity in the structure of the different corpora at the lexical sample task in different languages, we had found a heterogeneous vocabulary both in the XML tags and the attributes, forcing to develop 'ad hoc' parsers for each language. We missed a common and public document type definition for all the tasks.
Sense codification is another field where different solutions had been taken. In the English corpus nouns and adjectives are annotated using the WordNet 1.7.1. classification 1 (Fellbaum, 1998), while the verbs are based on Wordsmyth 2 (Scott, 1997). In the Catalan and Spanish tasks the sense inventory gives a more coarse-grained classification than WordNet. Both tasks have provided a dictionary with additional information as examples, typical collocations and the equivalent synsets at WordNet 1.5. Finally, the Italian sense inventory is based on the Multi-Wordnet dictionary 3 (Pianta et al., 2002). Unlike the other mentioned languages , the Italian task doesn't provide a separate file with the dictionary.
Besides the training data provided by Senseval, we have used the SemCor (Miller et al., 1993) collection in which every word is already tagged in its part of speech, sense and synset of WordNet.

Preprocessing
A tokenized version of the Catalan, Spanish and Italian corpora has been provided. In this version every word is tagged with its lemma and part of speech tag. This information has been manually annotated by human assessors both in the Catalan and the Spanish corpora. The Italian corpus has been processed automatically by the TnT POStagger 4 (Brants, 2000) including similar tags.
The English data lacked of this information, leading us to apply the TreeTagger 5 (Schmid, 1994) tool to the training and test data as a previous step to the disambiguation process.
Since the SemCor collection is already tagged, the preprocessing consisted in the segmentation of texts by the paragraph tag, obtaining 5382 different fragments. Each paragraph of Semcor has been used as a separate training example for the English lexical sample task. We applied the mapping provided by Senseval to represent verbs according to the verb inventory used in Senseval-3.

Approach
The supervised UNED WSD system is an exemplar based classifier that performs the disambiguation task measuring the similarity between a new instance and the representation of some labelled examples. However, instead of representing contexts as bags of terms and defining a similarity measure between the new context and the training contexts, we propose a representation of terms as bags of contexts and the definition of a similarity measure between terms. Thus, words, lemmas and senses are represented in the same space, where similarity measures can be defined between them. We call this space the Context Space. A new disambiguation context (bag of words) is transformed into the Context Space by the inner product, becoming a kind of abstract term suitable to be compared with singular senses that are represented in the same Context Space.

Representation
The training corpus is represented in the usual two-dimension matrix A as shown in Figure 1, where • c 1 , ..., c N is the set of examples or contexts in the training corpus. Contexts are treated as bags of words or lemmas.
• lem 1 , ..., lem T is the set of different words or lemmas in all the training contexts.
• w i,j is the weight for lem i in the training context c j .
A new instance q, represented with the vector of weights (w 1q , ..., w iq , ..., w T q ), is transformed into a vector in the context space q = (q 1 , ..., q j , ..., q N ), where q is given by the usual inner product q = q · A (Figure 1): If vectors c j (columns of matrix A) and vector q (original test context) are normalized to have a length equal to 1, then q j become the cosine between vectors q and c j . More formally, q = q.A = (cos(q, c 1 ), ..., cos(q, c j ), ..., cos(q, c N )) where At this point, both senses and the representation of the new instance q are represented in the same context space ( Figure 2) and a similarity measure can be defined between them: where sen ik is the k candidate sense for the ambiguous lemma lem i . Each component j of sen ik is set to 1 if lemma lem i is used with sense sen ik in the training context j, and 0 otherwise. For a new context of the ambiguous lemma lem i , the candidate sense with higher similarity is selected: Table 1 shows experimental results over the English Lexical Sample test of Senseval-3. System has been trained with the Senseval-3 data and the SemCor collection. The Senseval training data has been lemmatized and tagged with TreeTagger. Only nouns and adjectives have been considered in their canonical form.

Bag of words versus bag of contexts
Three different weights w ij have been tested: • Co-occurrence: w ij and w iq are set to {0,1} depending on whether lem i is present or not in context c j and in the new instance q respectively. After the inner product q · A, the components q j of q get the number of co-occurrences of different lemmas in both q and the training context c j .
• Term Frequency: w ij is set to tf ij , the number of occurrences of lem i in the context c j .
• tf.idf : w ij = (1 + log(tf ij )) · (log( N df i )), a standard tf.idf weight where df i is the number of contexts that contain lem i . These weights have been normalized ( w ij ||c j || ) and so, the inner product q·A generates a vector q of cosines as described above, where q j is the cosine between q and context c j .
Two similarity measures have been compared. The first one (maximum) is a similarity of q as bag of words with the training contexts of sense sen. The second one (cosine) is the similarity of sense sen with q in the context space: • Maximum: sim( sen, q) = = M ax N j=1 (sen j · q j ) = = M ax {j/sen∈c j } q j = = M ax {j/sen∈c j } cos(q, c j ) Similarity with sense sen is the highest similarity (cosine) between q (as bag of words) and each of the training contexts (as bag of words) for sense sen.
• Cosine: sim( sen, q) = cos( sen, Similarity with sense sen is the cosine in the Context Space between q and sen Table 1 shows that almost all the results are improved when the similarity measure (cosine) is applied in the Context Space. The exception is the consideration of co-ocurrences to disambiguate nouns. This exception led us to explore an alternative similarity measure aimed to improve results over nouns. The following sections describe this new similarity measure and the criteria underlying it.

Criteria for the similarity measure
Co-occurrences behave quite good to disambiguate nouns as it has been shown in the experiment above. However, the consideration of cooccurrences in the Context Space permits acumulative measures: Instead of selecting the candidate sense associated to the training context with the maximum number of co-occurrences, we can consider the co-occurences of q with all the contexts. The weights and the similarity function has been set out satisfying the following criteria: 1. Select the sense sen k assigned to more training contexts c i that have the maximum number of co-occurrences with the test context q. For example, if sense sen 1 has two training contexts with the highest number of co-occurrences and sense sen 2 has only one with the same number of co-occurrences, sen 1 must receive a higher value than sen 2 .
2. Try to avoid label inconsistencies in the training corpus. There are some training examples where the same ambiguous word is used with the same meaning but tagged with different sense by human assessors. Table 2 shows an example of this kind of inconsistencies.

Similarity measure
We assign the weights w ij and w iq to have q a vector of co-occurrences, where q j is the number of different nouns and adjectives that co-occurr in q and the training context c j . In this way, w ij is set to 1 if lem i is present in the context c j . Otherwise w ij is set to 0. Analogously for the new instance q, w iq is set to 1 if lem i is present in q and it is set to 0 otherwise. According to the second criterium, if there is only one context c 1 with the higher number of co-occurrences with q, then we reduce the value of this context by reducing artificially its number of co-occurrences: Being c 2 a context with the second higher number of cooccurrences with q, then we assign to the first context c 1 the number of co-occurrences of context c 2 .
Comparing results in both tables we observe that the new similarity measure only behaves better for the disambiguation of nouns. However, the difference is big enough to improve overall results. The application of the second criterium (try to avoid label inconsistencies) also improves the results as shown in Tables 3  and 4. Table 4 shows the effect of applying this second criterium to all the languages we have participated in. With the exception of Catalan, all results are improved slightly (about 1%) after the filtering of singular labelled contexts. Although it is a regular behavior, this improvement is not statistically significative.  The results submited to Senseval-3 were generated with the system described in Section 4.4.
Since one sense is assigned to every test context, precison and recall have equal values. Ta Table 5: Official results at Senseval-3, precisionrecall Differences between languages are quite remarkable and show the system dependence on the training corpora and the sense inventory.
In the English task, 16 test instances have a correct sense not present in the training corpus. Since we don't use the dictionary information our system was unable to deal with none of them. In the same way, 68 test instances have been tagged as "Unasignable" sense and again the system was unable to detect none of them.

Conclusion and work in progress
We have shown the exemplar-based WSD system developed by UNED for the Senseval-3 lexical sample tasks. The general approach is based on the definition of a context space that becomes a flexible tool to prove quite different similarity measures between training contexts and new instances. We have shown that standard similarity measures improve their results applied inside this context space. We have established some criteria to instantiate this general approach and the resulting system has been evaluated at Senseval-3. The new similarity measure improves the disambiguation of nouns and obtains better overall results. The work in progress includes: • the study of new criteria to lead us to alternative measures, • the development of particular disambiguation strategies for verbs, nouns and adjectives, • the inclusion of the dictionary information, and • the consideration of WordNet semantic relationships to extend the training corpus.