{'fraction_non_alphanumeric': 0.028753301866366432, 'fraction_numerical': 0.0037240722296799896, 'mean_word_length': 4.564819277108434, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Antonymy and Conceptual Vectors

For meaning representations in NLP, we focus our attention on thematic aspects and conceptual vectors. The learning strategy of conceptual vectors relies on a morphosyntaxic analysis of human usage dictionary definitions linked to vector propagation. This analysis currently doesn't take into account negation phenomena. This work aims at studying the antonymy aspects of negation, in the larger goal of its integration into the thematic analysis. We present a model based on the idea of symmetry compatible with conceptual vectors. Then, we define antonymy functions which allows the construction of an antonymous vector and the enumeration of its potentially antinomic lexical items. Finally, we introduce a measure which evaluates how a given word is an acceptable antonym for a term.


Introduction
Research in meaning representation in NLP is an important problem still addressed through several approaches. The NLP team at LIRMM currently works on thematic and lexical disambiguation text analysis (Laf01). Therefore we built a system, with automated learning capabilities, based on conceptual vectors for meaning representation. Vectors are supposed to encode 'ideas' associated to words or to expressions. The conceptual vectors learning system automatically defines or revises its vectors according to the following procedure. It takes, as an input, definitions in natural language contained in electronic dictionaries for human usage. These definitions are then fed to a morphosyntactic parser that provides tagging and analysis trees. Trees are then used as an input to a procedure that computes vectors using tree geometry and syntactic functions. Thus, a kernel of manually indexed terms is necessary for bootstrapping the analysis. The transversal relationships 1 , such as synonymy (LP01), antonymy and hyperonymy, that are more or less explicitly mentioned in definitions can be used as a way to globally increase the coherence of vectors. In this paper, we describe a vectorial function of antonymy. This can help to improve the learning system by dealing with negation and antonym tags, as they are often present in definition texts. The antonymy function can also help to find an opposite thema to be used in all generative text applications: opposite ideas research, paraphrase (by negation of the antonym), summary, etc.

Conceptual Vectors
We represent thematic aspects of textual segments (documents, paragraph, syntagms, etc) by conceptual vectors. Vectors have been used in information retrieval for long (SM83) and for meaning representation by the LSI model (DDL + 90) from latent semantic analysis (LSA) studies in psycholinguistics. In computational linguistics, (Cha90) proposes a formalism for the projection of the linguistic notion of semantic field in a vectorial space, from which our model is inspired. From a set of elementary concepts, it is possible to build vectors (conceptual vectors) and to associate them to lexical items 2 . The hypothesis 3 that considers a set of concepts as a generator to language has been long described in (Rog52). Polysemic words combine different vectors corresponding to different meanings. This vector approach is based on known mathematical properties, it is thus possible to undertake well founded formal manipulations attached to reasonable linguistic interpretations. Concepts are defined from a thesaurus (in our prototype applied to French, we have chosen (Lar92) where 873 concepts are identified). To be consistent with the thesaurus hypothesis, we consider that this set constitutes a generator family for the words and their meanings. This familly is probably not free (no proper vectorial base) and as such, any word would project its meaning on it according to the following principle. Let be C a finite set of n concepts, a conceptual vector V is a linear combinaison of elements c i of C. For a meaning A, a vector V (A) is the description (in extension) of activations of all concepts of C. For example, the different meanings of door could be projected on the following concepts (the In practice, the larger C is, the finer the meaning descriptions are. In return, the computing is less easy: for dense vectors 4 , the enumeration of activated concepts is long and difficult to evaluate. We prefer to select the thematically closest terms, i.e., the neighbourhood. For instance, the closest terms ordered by increasing distance to door are: V( door )= portal , portiere , opening , gate , barrier ,. . .

Angular Distance
Let us define Sim(A, B) as one of the similarity measures between two vectors A et B, often used in information retrieval (Mor99). We can express this function as: with "·" as the scalar product. We suppose here that vector components are positive or null. Then, we define an angular distance D A between two vectors A and B as D A (A, B) = arccos(Sim (A, B)). Intuitively, this function constitutes an evaluation of the thematic proximity and measures the angle between the two vectors. We would generally consider that, for a distance D A (A, B) ≤ π (45 degrees) A and B are thematically close and share many concepts. For D A (A, B) ≥ π 4 , the thematic proximity between A and B would be considered as loose. Around π 2 , they have no relation. D A is a real distance function. It verifies the properties of reflexivity, symmetry and triangular inequality. We have, for example, the following angles(values are in radian and degrees).
The first one has a straightforward interpretation, as a tit cannot be closer to anything else than itself. The second and the third are not very surprising since a tit is a kind of sparrow which is a kind of bird . A tit has not much in common with a train , which explains a large angle between them. One can wonder why there is 32 degrees angle between tit and insect , which makes them rather close. If we scrutinise the definition of tit from which its vector is computed (Insectivourous passerine bird with colorful feather.) perhaps the interpretation of these values seems clearer. In effect, the thematic is by no way an ontological distance.

Conceptual Vectors Construction.
The conceptual vector construction is based on definitions from different sources (dictionaries, synonym lists, manual indexations, etc). Definitions are parsed and the corresponding conceptual vector is computed. This analysis method shapes, from existing conceptual vectors and definitions, new vectors. It requires a bootstrap with a kernel composed of pre-computed vectors. This reduced set of initial vectors is manually indexed for the most frequent or difficult terms. It constitutes a relevant lexical items basis on which the learning can start and rely. One way to build an coherent learning system is to take care of the semantic relations between items. Then, after some fine and cyclic computation, we obtain a relevant conceptual vector basis. At the moment of writing this article, our system counts more than 71000 items for French and more than 288000 vectors, in which 2000 items are concerned by antonymy. These items are either defined through negative sentences, or because antonyms are directly in the dictionnary. Example of a negative definition:

Definition and Characterisation of Antonymy
We propose a definition of antonymy compatible with the vectorial model used. Two lexical items are in antonymy relation if there is a symmetry between their semantic components relatively to an axis. For us, antonym construction depends on the type of the medium that supports symmetry. For a term, either we can have several kinds of antonyms if several possibilities for symmetry exist, or we cannot have an obvious one if a medium for symmetry is not to be found. We can distinguish different sorts of media: (i) a property that shows scalar values (hot and cold which are symmetrical values of temperature), (ii) the true-false relevance or application of a property (e.g. existence/nonexistence) (iii) cultural symmetry or opposition (e.g. sun/moon).From the point of view of lexical functions, if we compare synonymy and antonymy, we can say that synonymy is the research of resemblance with the test of substitution (x is synonym of y if x may replace y), antonymy is the research of the symmetry, that comes down to investigating the existence and nature of the symmetry medium. We have identified three types of symmetry by relying on (Lyo77), (Pal76) and (Mue97). Each symmetry type characterises one particular type of antonymy. In this paper, for the sake of clarity and precision, we expose only the complementary antonymy. The same method is used for the other types of antonymy, only the list of antonymous concepts are different.

Complementary Antonymy
The complementary antonyms are couples like event/unevent, presence/absence.
In logical terms, we would have: This corresponds to the exclusive disjunction relation. In this frame, the assertion of one of the terms implies the negation of the other. Complementary antonymy presents two kinds of symmetry, (i) a value symmetry in a boolean system, as in the examples above and (ii) a symmetry about the application of a property (black is the absence of color, so it is "opposed" to all other colors or color combinaisons).

Principles and Definitions.
The aim of our work is to create a function that would improve the learning system by simulating antonymy. In the following, we will be mainly interested in antonym generation, which gives a good satisfaction clue for these functions. We present a function which, for a given lexical item, gives the n closest antonyms as the neighbourhood function V provides the n closest items of a vector. In order to know which particular meaning of the word we want to oppose, we have to assess by what context meaning has to be constrained. However, context is not always sufficient to give a symmetry axis for antonymy. Let us consider the item father . In the family context, it can be opposite to mother or to children being therefore ambiguous because mother and children are by no way similar items. It should be useful, when context cannot be used as a symmetry axis, to refine the context with a conceptual vector which is considered as the referent. In our example, we should take as referent filiation , and thus the antonym would be children or the specialised similar terms (e.g. sons , daughters ) marriage or masculine and thus the antonym would be mother .
The function AntiLex S returns the n closest antonyms of the word A in the context defined by C and in reference to the word R. The partial function AntiLex R has been defined to take care of the fact that in most cases, context is enough to determine a symmetry axis. AntiLex B is defined to determine a symmetry axis rather than a context. In practice, we have AntiLex B = AntiLex R . The last function is the absolute antonymy function. For polysemic words, its usage is delicate because only one word defines at the same time three things: the word we oppose, the context and the referent. This increases the probability to get unsatisfactory results. However, given usage habits, we should admit that, practically, this function will be the most used. It's sequence process is presented in picture 1. We note Anti(A,C) the Items without antonyms: it is the case of material objects like car, bottle, boat, etc. The question that raises is about the continuity the antonymy functions in the vector space. When symmetry is at stake, then fixed points or plans are always present. We consider the case of these objects, and in general, non opposable terms, as belonging to the fixed space of the symmetry. This allows to redirect the question of antonymy to the opposable properties of the concerned object. For instance, if we want to compute the antonym of a motorcycle , which is a ROAD TRANSPORT, its opposable properties being NOISY and FAST, we consider its category (i.e. ROAD TRANSPORT ) as a fixed point, and we will look for a road transport (SILEN-CIOUS and SLOW ), something like a bicycle or an electric car . With this method, thanks to the fixed points of symmetry, opposed "ideas" or antonyms, not obvious to the reader, could be discovered.

Antonym vectors of concept lists
Anti functions are context-dependent and cannot be free of concepts organisation. They need to identify for every concept and for every kind of antonymy, a vector considered as the opposite. We had to build a list of triples concept, context, vector . This list is called antonym vectors of concept list (AVC).

AVC construction.
The Antonym Vectors of Concepts list is manually built only for the conceptual vectors of the generating set. For any concept we can have the antonym vectors such as: As items, concepts can have, according to the context, a different opposite vector even if they are not polysemic. For instance, DE-STRUCTION can have for antonyms PRESERVA-TION, CONSTRUCTION, REPARATION or PROTEC-TION. So, we have defined for each concept, one conceptual vector which allows the selection of the best antonym according to the situation. For example, the concept EXISTENCE has the vector NON-EXISTENCE for antonym for any context. The concept DISORDER has the vector of ORDER for antonym in a context constituted by the vectors of ORDER ⊕DISORDER 5 and has CLAS-SIFICATION in a context constituted by CLASSI-FICATION and ORDER.
The function AntiC(C i , V context ) returns for a given concept C i and the context defined by V context , the complementary antonym vector in the list.

Construction of the antonym vector: the Anti Function 4.3.1 Definitions
We define the relative antonymy function Anti R (A, C) which returns the opposite vector of A in the context C and the absolute antonymy function Anti A (A) = Anti R (A, A). The usage of Anti A is delicate because the lexical item is considered as being its own context. We will see in 4.4.1 that this may cause real problems because of sense selection. We should stress now on the construction of the antonym vector from two conceptual vectors: V item , for 5 ⊕ is the normalised sum V = A ⊕ B | vi = x i +y i V the item we want to oppose and the other, V c , for the context (referent).

Construction of the Antonym Vector
The method is to focus on the salient notions in V item and V c . If these notions can be opposed then the antonym should have the inverse ideas in the same proportion. That leads us to define this function as follows: We crafted the definition of the weight P after several experiments. We noticed that the function couldn't be symmetric (we cannot reasonably have Anti R (V( hot ),V( temperature )) = Anti R (V( temperature ),V( hot ))). That is why we introduce this power, to stress more on the ideas present in the vector we want to oppose. We note also that the more conceptual 6 the vector is, the more important this power should be. That is why the power is the variation coefficient 7 which is a good clue for "conceptuality". To finish, we introduce this function max because an idea presents in the item, even if this idea is not present in the referent, has to be opposed in the antonym. For example, if we want the antonym of cold in the temperature context, the weight of cold has to be important even if it is not present in temperature .

Lexical Items and Vectors:
Problem and Solutions The goal of the functions AntiLex is to return antonym of a lexical item. They are defined with the Anti function. So, we have to use tools which allow the passage between lexical items and vectors. This transition is difficult because of polysemy, i.e. how to choose the right relation between an item and a vector. In other words, how to choose the good meaning of the word.

Transition lexical items →
Conceptual Vectors As said before, antonymy is relative to a context. In some cases, this context cannot be sufficient to select a symmetry axis for antonymy.
To catch the searched meaning of the item and, if it is different from the context, to catch the selection of the meaning of the referent, we use the strong contextualisation method. It computes, for a given item, a vector. In this vector, some meanings are favoured against others according to the context. Like this, the context vector is also contextualised.
This contextualisation shows the problem caused by the absolute antonymy function Anti α R . In this case, the method will compute the vector of the word item in the context item. This is not a problem if item has only one definition because, in this case, the strong contextualisation has no effect. Otherwise, the returned conceptual vector will stress on the main idea it contains which one is not necessary the appropriate one.

Transition Conceptual Vectors →
Lexical Items This transition is easier. We just have to compute the neighbourhood of the antonym vector V ant to obtain the items which are in thematic antonymy with V item . With this method, we have, for instance: It is not important to contextualise the concept LIFE because we can consider that, for every context, the opposite vector is the same. In complementary antonymy, the closest item is DEATH. This result looks satisfactory. We can see that the distance between the antonymy vector and DEATH is not null. It is because our method is not and cannot be an exact method. The goal of our function is to build the best (closest) antonymy vector it is possible to have. The construction of the generative vectors is the second explanation. Generative vectors are interdependent. Their construction is based on an ontology. To take care of this fact, we don't have boolean vectors, with which, we would have exactly the same vector. The more polysemic the term is, the farthest the closest item is, as we can see it in the first two examples.
We cannot consider, even if the potential of antonymy measure is correct, the closest lexical item from V anti as the antonym. We have to consider morphological features. Simply speaking, if the antonym of a verb is wanted, the result would be better if a verb is caught.

Antonymy Evaluation Measure
Besides computing an antonym vector, it seems relevant to assess wether two lexical items can be antonyms. To give an answer to this question, we have created a measure of antonymy evaluation. Let A and B be two vectors. The question is precisely to know if they can reasonably be antonyms in the context of C. The antonymy measure M anti Eval is the angle between the sum of A and B and the sum of Anti c R (A, C) and Anti c R (B, C). Thus, we have: The antonymy measure is a pseudo-distance. It verifies the properties of reflexivity, symmetry and triangular inequality only for the subset of items which doesn't accept antonyms. In this case, notwithstanding the noise level, the measure is equal to the angular distance. In the general case, it doesn't verify reflexivity. The conceptual vector components are positive and we have the property: Dist anti ∈ [0, π 2 ]. The smaller the measure, the more 'antonyms' the two lexical items are. However, it would be a mistake to consider that two synonyms would be at a distance of about π 2 . Two lexical items at π 2 have not much in common 8 . We would rather see here the illustration that two antonyms share some ideas, specifically those which are not opposable or those which are opposable with a strong activation. Only specific activated concepts would participate in the opposition. A distance of π 2 between two items should rather be interpreted as these two items do not share much idea, a kind of anti-synonymy. This result confirms the fact that antonymy is not the exact inverse of synonymy but looks more like a 'negative synonymy' where items remains quite related. To sum up, the antonym of w is not a word that doesn't share ideas with w, but a word that opposes some features of w.

Examples
In the following examples, the context has been ommited for clarity sake. In these cases, the context is the sum of the vectors of the two items. The above examples confirm what presented. Concepts EXISTENCE and NON-EXISTENCE are very strong antonyms in complementary antonymy. The effects of the polysemy may explain that the lexical items existence and non-existence are less antonyms than their related concepts. In complementary antonymy, CAR is its own antonym. The antonymy measure between CAR and EXISTENCE is an example of our previous remark about vectors sharing few ideas and that around π/2 this measure is close to the angular distance (we have D A (existence, car) = 1.464.). We could consider of using this function to look in a conceptual lexicon for the best antonyms. However, the computation cost (around a minute on a P4 at 1.3 GHz) would be prohibitive.

Action on learning and method evaluation
The function is now used in the learning process. We can use the evaluation measure to show the increase of coherence between terms: There is no change in concepts because they are not learned. In the opposite, the antonymy evaluation measure is better on items. The exemple shows that existence and non-existence have been largely modified. Now, the two items are stronger antonyms than before and the vector basis is more coherent. Of course, we can test these results on the 71000 lexical items which have been modified more or less directly by the antonymy function. We have run the test on about 10% of the concerned items and found an improvement of the angular distance through M anti EvalC ranking to 0.1 radian.

Conclusion
This paper has presented a model of antonymy using the formalism of conceptual vectors. Our aim was to be able: (1) to spot antonymy if it was not given in definition and thus provide an antonym as a result, (2) to use antonyms (discovered or given) to control or to ensure the coherence of an item vector, build by learning, which could be corrupted. In NLP, antonymy is a pivotal aspect, its major applications are thematic analysis of texts, construction of large lexical databases and word sense disambiguation. We grounded our research on a computable linguisitic theory being tractable with vectors for computational sake. This preliminary work on antonymy has also been conducted under the spotlight of symmetry, and allowed us to express antonymy in terms of conceptual vectors. These functions allow, from a vector and some contextual information, to compute an antonym vector. Some extensions have also been proposed so that these functions may be defined and usable from lexical items. A measure has been identified to assess the level of antonymy between two items. The antonym vector construction is necessary for the selection of opposed lexical items in text generation. It also determines opposite ideas in some negation cases in analysis. Many improvements are still possible, the first of them being revision of the VAC lists. These lists have been manually constructed by a reduced group of persons and should widely be validated and expanded especially by linguists. We are currently working on possible improvements of results through learning on a corpora.