{'fraction_non_alphanumeric': 0.023941136923881376, 'fraction_numerical': 0.008665122880406365, 'mean_word_length': 4.88590899098703, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automatically Building a Multilingual Lexicon of False Friends With No Supervision

Cognate words, defined as words in different languages which derive from a common etymon, can be useful for language learners, who can leverage the orthographical similarity of cognates to more easily understand a text in a foreign language. Deceptive cognates, or false friends, do not share the same meaning anymore; these can be instead deceiving and detrimental for language acquisition or text understanding in a foreign language. We use an automatic method of detecting false friends from a set of cognates, in a fully unsupervised fashion, based on cross-lingual word embeddings. We implement our method for English and five Romance languages, including a low-resource language (Romanian), and evaluate it against two different gold standards. The method can be extended easily to any language pair, requiring only large monolingual corpora for the involved languages and a small bilingual dictionary for the pair. We additionally propose a measure of “falseness” of a false friends pair. We publish freely the database of false friends in the six languages, along with the falseness scores for each cognate pair. The resource is the largest of the kind that we are aware of, both in terms of languages covered and number of word pairs.


Introduction
Cognates are words in genetically related languages (languages descending from a common ancestor) with a common proto-word. For example, the Romanian word victorie and the Italian word vittoria are cognates, as they both descend from the Latin word victoria (meaning victory)see Figure 1. Cognates can be useful for language learners, who can have a prior understanding of what a word in a foreign language means only through knowing its cognate: a Romanian speaker can easily learn the Italian word vittoria, and, for related languages where much of the vocabulary consists of cognates, non-native speakers of the related language can even achieve basic understanding of texts in the foreign language without ever having studied it. In most cases, cognates have preserved similar meanings across languages, but there are also exceptions. In some cases, the meanings of cognates has diverged from the common etymon through their use in each of the two languages, and their meanings became different from each other. These are called deceptive cognates or, more commonly, false friends. Here we use the definition of cognates that refers to words with similar appearance and some common etymology, while true cognates is used to refer to cognates which also have a common meaning, and deceptive cognates or false friends refers to cognate pairs which do not have the same meaning (anymore).  Dominguez and Nerlich (2002) distinguish between chance false friends, which have similar form but different etymologies as well as different meanings in different languages, and semantic false friends, which share the etymological origin, but their meanings differ (to some extent) in different languages. In this study we focus on semantic false friends (or deceptive cognates), by leveraging a previously published resource of cognate words extracted from etymological dictionaries. Semantic false friends are more common and often more difficult to correctly detect, nevertheless the same method can in principle be used to identify chance false friends. Many false friends have diverged into entirely different meanings. There are many examples, however, for which the changes in meaning are more subtle, at the level of connotations, and more difficult to detect even for humans. The notion of semantic equivalence used to define false friends is in itself ambiguous and difficult to treat as a binary property, and we propose in this paper that the quality of a cognate pair of being in a false friends relationship should also be treated as a spectrum. According to Uban et al. (2019), a hard false friend is a pair of cognates for which the meanings of the two words have diverged enough such that they no longer have the same meaning, and should not be used interchangibly (as translations of one another). In this category fall most known examples of false friends, such as the French-English cognate pair attendre / attend: in French, attendre has a completely different meaning than in English, which is to wait. A different and more subtle type of false friends can result from more minor semantic shifts between the cognates. In such pairs, the meaning of the cognate words may remain roughly the same, but with a difference in nuance or connotation. Such an example is the Romanian-Italian cognate pair amic / amico. Here, both cognates mean friend, but in Italian the conotation is that of a closer friend, whereas the Romanian amic denotes a more distant friend, or even aquaintance. A more suitable Romanian translation for am-ico would be prieten, while a better translation in Italian for amic could be conoscente.
In the case of soft false friends, their meaning is roughly the same (and since they can be considered "partial" translations, they can even be found in bilingual dictionaries as correct translations), nevertheless, translating one word for the other would be an inaccurate use of the language. In these cases, instead of helping non-natives to more easily understand a text in a foreign language, cognates can instead cause more confusion and deceive the language learner into misunderstanding the text. This is why having a method to correctly detect even the more subtle kind of false friends is very important for assisting with language learning and text comprehension in a foreign language. Moreover, identifying false friends can be useful not only for language acquisition, but also in downstream applications relying on cognates, such as machine translation.
In this paper we propose using a fully automatic and unsupervised algorithm in order to detect false friends, and we generate a lexicon of false friends, along with falseness scores for each pair, for every language pair among six languages considered (Romance languages and English). Our method is based on the false friend detection algorithm relying on cross-lingual word embeddings introduced in Uban et al. (2019), to which we add a more extensive evaluation of the resulted false friends pairs, including the extended list of over 3,000 cognate sets (instead of the smaller 305 words list evaluated in the previous study) and additionally include an evaluation and analysis of the falseness measure. We publish freely the resulted database comprised of false friend pairs for each pair of languages considered, and the falseness score for each pair.

Related Work
A comprehensive list of cognates and false friends for every language pair is difficult to find and expensive to manually build. Moreover, dictionaries grow outdated and it is difficult to continuously update them to incorporate new words in the vocabulary. This is why applications have to rely on automatically identifying false friends. There have been a number of previous studies attempting to automatically extract pairs of true cognates and false friends from corpora or from dictionaries. Most methods are based either on orthographic and phonetic similarity (Inkpen et al., 2005), or require large parallel corpora (Nakov et al., 2009) or dictionaries (St Arnaud et al., 2017). Inkpen et al. (2005) use orthographic features to extract cognate pairs for French-English, but do not take semantic similarity into account. Torres and Aluísio (2011) also rely on orthographic and phonetic features, to which they add a semantic feature extracted from a bilingual dictionary. They additionally release a lexicon of Spanish-Portuguese false friends and true cognates, obtained through manual annotation, that they use to evaluate their algorithms. Nakov et al. (2009) identify false friends pairs in Bulgarian and Russian by making use of sentence-aligned parallel corpora. In (Aminian et al., 2015) the authors propose using a model of identifying false friends from parellel corpora in order to improve English-Egyptian statistical machine translation.
Cross-lingual semantic word similarity consists in identifying words that refer to similar semantic concepts and convey similar meanings across languages (Vulic and Moens, 2013). Some of the most popular approaches rely on probabilistic models (Vulic and Moens, 2014) and cross-lingual word embeddings (Søgaard et al., 2017). There have been few previous studies using word embeddings for the detection of false friends or cognate words, usually using simple methods on only one or two pairs of languages (Castro et al., 2018;Torres and Aluísio, 2011). Castro et al. (2018) detect false friends in Spanish-Portuguese, employing a classifier that learns from features extracted from multilingual embedding spaces. In (Mitkov et al., 2007) the authors use a method based on distributed representations of words in a continuous space built using comparable corpora, as well as a taxonomy-based approach, to identify false friends in four language pairs involving English, French, German and Spanish. Uban et al. (2019) propose a method for identifying and correcting false friends, as well as define a measure of their "falseness", using cross-lingual word embeddings. We base our study on the method proposed here.

Detecting False Friends
In the following section we describe the algorithm used for detecting false friends automatically, in an unsupervised manner, based on a seed set of cognate sets.

Cross-lingual Word Embeddings
Word embeddings are vectorial representations of words in a continuous space, built by training a model to predict the occurrence of a target word in a text corpus given its context. Based on the distributional hypothesis stating that similar words occur in similar contexts, these vectorial representations can be seen as semantic representations of words and can be used to compute semantic similarity between word pairs (representations of words with similar meanings are expected to be close together in the embeddings space). In our study we make use of word embeddings computed using the FastText algorithm, pre-trained on Wikipedia for the six languages in question. The vectors have 300 dimensions, and were obtained using the skip-gram model described by Bojanowski et al. (2016) with default parameters. These pre-trained embeddings are suitable for our multilingual study since: they are trained on large amounts of text, which minimizes the amount of noise in the vectors, making them good approximators of word meanings; and they are trained on text that is relatively uniform in style and topic -ensuring any differences in the structure of the embedding spaces of different languages is dependent on the language, rather than an artifact of the topic or genre of the corpus. Nevertheless, even high quality embeddings can be noisy or biased and this should be kept in mind when interpreting the results of our experiments.
To compute the semantic divergence of cognates across sister languages, we need to obtain a multilingual semantic space, which is shared between the cognates. Having the representations of both cognates in the same semantic space, we can then compute the semantic distance between them using their vectorial representations in this space. For a given pair of languages among the six considered, we can then accomplish this following the steps below: 1.
Step 1. Train a word embeddings model for each of the two languages. 2.
Step 2. Obtain a shared embedding space, common to the two languages. This is accomplished using an alignment algorithm, which consists of finding a linear transformation between the two spaces that on average optimally transforms each vector in one embedding space into a vector in the second embedding space, minimizing the distance between a few seed word pairs (which are assumed to have the same meaning), based on a small bilingual dictionary. The linear nature of the transformation guarantees distances between words in the original spaces (within each language) are preserved. For our purposes, we use the publicly available FastText multilingual word embeddings pre-aligned in a common vector space (Conneau et al., 2017). 1 3.
Step 3. Compute the semantic distance for the pair of cognates in the two languages, using a vectorial distance (we chose cosine distance) on their corresponding vectors in the shared embedding space.

Cognates Dataset
In order to identify false friends (deceptive cognates), we start from a database of cognates, defined as words with common etymology and similar orthography. As our data source, we use the list of cognate sets in Romance languages published in (Ciobanu and Dinu, 2014). It contains 3,218 complete cognate sets in Romanian, French, Italian, Spanish and Portuguese, along with their Latin common ancestors, extracted from online etymology dictionaries. A subset of 305 of these sets also contains the corresponding cognate (in the broad sense, since these are mostly borrowings) in English.
One complete example of a cognate set for the word "architect" in the Romance languages is illustrated in Table 1.

Deceptive Cognates and Falseness
The multilingual embedding spaces as defined above can be used to measure the semantic distances between cognates in order to detect pairs of false friends, which are simply defined as pairs of cognates which do not share the same meaning. More specifically, following the false friends detection and correction algorithm in Uban et al. (2019), we consider a pair of cognates to be a false friend pair if in the shared semantic space, there exists a word in the second language which is semantically closer to the original word than its cognate in that language (in other words, the cognate is not the optimal translation). The arithmetic difference between the semantic distance between these words and the semantic distance between the cognates will be used as a measure of the falseness of the false friend. The 1 https://github.com/facebookresearch/MUSE closest neighbor of the target word in the second language's embedding space is considered to be the correct semantic equivalent of the target word in the second language, and can be provided as a "correction" for the false friend. The algorithm for detecting (and correcting) false friends, as well as measuring their degree of falseness, can be described as shown in Algorithm 1.
Algorithm 1 Detection and correction of false friends 1: Given the cognate pair (c 1 , c 2 ) where c 1 is a word in lang 1 and c 2 is a word in lang 2 : 2: Find the word w 2 in lang 2 such that for any w i in lang 2 , distance(c 1 , w 2 ) < distance(c 1 , w i ) 3: if w 2 = c 2 then 4: (c 1 , c 2 ) is a pair of false friends 5: Degree of falseness = distance(c 1 , w 2 ) − distance(c 1 , c 2 ) 6: return w 2 as potential correction 7: end if

False Friends Dataset
We use the algorithm described in the previous section to build a database of false friends pairs for each language pair among the six languages considered, which we make freely available 2 . False friends for Romance languages are extracted from the original 3,218 cognate sets, resulting in 500 to 1,200 of detected false friends for each language pair. For English, the original cognate resource contains a smaller set of only 305 cognate sets, which results in smaller false friends lists for language pairs involving English. Table 2 shows the number of false friends pairs generated for each language pair, and included in the published resource.

Evaluation
In order to evaluate the quality of the false friends dataset generated with our algorithm, we first test its accuracy against a multilingual dictionary, for this study we choose to use Open Multilingual WordNet (Miller, 1998;?). A pair of words with common etymology are considered true cognates if they belong to the same WordNet synset (are synonyms), and false friends if they are not synonyms. Using this standard, the obtained measured accuracy is between 73% and 81%, depending on the language pair considered. Table 3 presents a breakdown of the obtained performance per language pair considered. Romanian is the only language missing from the evaluation since it is not represented in multilingual WordNet. Since English is only available for a subset of the cognates, evaluation for Romance languages may be more robust.
We select a few results of the algorithm to show in Ta Table 3: Performance for all language pairs using WordNet as gold standard.
/ tenero (meaning tender), with the Italian correction giovane (young), or inimȃ (heart) / anima (soul), corrected to cuore (heart). The falseness scores also reflect the degree of semantic drift between the false friends, with the tânȃr/tenero pair being more dissimilar than inimȃ/anima. The amic/amico/amichetto set, which refers to different degrees of friendship, is awarded the lowest falseness score. It is valuable to note the algorithm also selects word pairs which can technically be considered true cognates (long/luengo -meaning long), but are not used as such in current speech: largo is more frequently used than luengo. This is to be expected since the algorithm is based on word usage in language (since this is the basis of the embedding training algorithm). We also illustrate an example where the algorithm makes a mistake: in the case of stânga (left)/stanco (tired), the algorithm rightly identifies this as a false friends pair, but provides an erroneous correction: destra is the Italian word for left, not right. This error can also be traced back to the nature of semantic similarity as captured by word embeddings: related but not equivalent words (and sometimes even antonyms) can have similar embedding vectors due to their similar occurence patterns in corpora.  In a second experiment, we measure accuracy of false friend detection on a manually curated list of false friends and true cognates in Spanish and Portuguese, used in a previous study (Castro et al., 2018), and introduced in (Torres and Aluísio, 2011). This resource is composed by 710 Spanish-Portuguese word pairs: 338 true cognates and 372 false friends. We also compare our results to the ones reported in this study, which uses a method similar to ours (using a simple classifier that takes embedding similarities as features to identify false friends) and shows improvements over results in previous research. The results are shown in Table 5. We also compute the same metrics using a falseness threshold as a lower bound to decide whether two words are false friends; results show a trade-off between recall and precision when using a threshold. The following section discusses in more detail the use of falseness thresholds.
In this second experiment, WordNet is used as a baseline algorithm for false friend identification instead of a gold standard. Its relatively poor results as compared to the automatic methods may stem from the lower coverage it has as compared to corpus-based methods. Castro et al. (2018) show that only 55% of the word pairs in the evaluation set used here are found in WordNet synsets. This shows that using WordNet as an evaluation standard has its limits, and that corpus-based methods such as the one we propose have an advantage over dictionary-based methods.

Falseness as a Spectrum
The measure of falseness that we provide for every detected pair of false friends can be useful not only for a better un-   derstanding of the linguistic phenomenon behind the semantic divergence of the cognates, but also for a more flexible integration with downstream applications. When our resource of false friends, a custom threshold of falseness could be set and used to filter out false friends in a more coarse or fine-grained way, depending on the needs of the application. For example, for applications where capturing subtle changes in meaning is important, maintaining a low threshold of falseness is useful. On the other hand, when simply identifying false friends which have entirely different meanings, choosing a high threshold may be sufficient and might also ensure a lower rate of false positives, by filtering out the delicate cases of false friend pairs with similar meanings, which lie at the boundary between true and deceptive cognates.
We perform an analysis of the effect of varying the threshold of falseness used to discriminate between true cognates and false friends, by re-evaluating the generated false friends against WordNet using a varying threshold of falseness to discriminate between false friends and true cognates (as opposed to the simple evaluation in the previous chapter, where no threshold was set, which is equivalent to using a falseness threshold of 0). In this way, we are able to discover the optimal falseness threshold to use in order to maximize performance relative to the WordNet standard. We choose the threshold which leads to maximum average accuracy across all language pairs on a separate training set of 80% of the word pairs. The rest of 20% of word pairs are used to evaluate the method now employing a falseness threshold, set to the optimal value according to the training phase. The optimal falseness threshold f t is found to Falseness threshold 0 (None) 0.2 (optimal) Accuracy 80.57 85.85 Table 6: Best overall accuracy of our method be 0.2, and the average overall accuracy with this threshold is 85.85%. Table 6 shows the difference in accuracy when using the optimal threshold, and Figure 2 illustrates the variation of accuracy, precision and recall (on average for all language pairs) when varying the threshold between 0 and 1.
where f t is a falseness threshold and LP is the set of all language pairs: (2) The fact that in WordNet the optimal falseness is positive means that many of the extracted false friends with very low falseness make up for most of false positives (actual true cognates), which are not identified as such not necessarily because they are actually different in meaning, but rather because of artifacts of the embedding space. We then perform the same experiment, but this time evaluate using the curated cognates sets in Spanish-Portuguese. In this case, the optimal threshold is found to be 0.1, and the threshold of 0.2 found in the previous experiment leads to worse results than not using a threshold at all. In order to confirm the difference stems from the the different definition of cross-lingual synonymy in the two datasets and is not specific to just the language pair, we compute the optimal falseness threshold relative to WordNet specifically for Spanish-Portuguese and find an optimal falseness of 0.3. This difference between the optimal threshold values for the two different gold standards suggests the two resources were built with different assumptions about meaning equivalence, and confirms that the availability of the falseness measure can be useful for tuning the false friend detection algorithm to the specific task and standards of the particular application.

Error Analysis and Discussion
As suggested in the previous section, a significant source of error relative to the WordNet standard are low-falseness pairs of detected false friends. Figures 3 and 4 show the distribution of falseness scores across all word pairs in all languages. We separately show the distribution of false friends extracted with our method that were evaluated as actual false friends using WordNet, and the pairs of extracted false friends that are actually true cognates according to Word-Net. The much lower falseness values for word pairs in the second category (false positives in the evaluation using WordNet) suggest that many of the false positives produced by the algorithm fall in the range of word pairs with very subtle differences in meaning. These might stem from imperfections in the embedding space or from the too strong  Table 4 illustrate these types of errors, such is the case of the previously discussed pair stânga / stanco, with the mistaken correction destra. More subtle inaccuracies can consist for example of mismatched parts of speech: such as the case of change (noun) / caer (infinitive verb) / cambia (indicative verb). On the other hand, we believe including the falseness score in the dataset can be useful precisely to remedy this issue when needed, and that in some cases, these low-falseness word pairs could even be considered actual false friends (rather than errors of classification) by a standard of meaning equivalence that is more strict than the one used in WordNet's synsets.
A second source of errors is found in the original cognates data source that we use to then discriminate into true and false cognates. Since it is also an automatically built resource, some of the word pairs are falsely labelled as cognates, and may further perpetuate into false positives in our algorithm.

Conclusions
We have built and made freely available a database of false friends in six languages, and evaluated it against WordNet and against a manually curated dataset of false friends, obtaining state of the art results. To the best of our knowledge, the published database is the largest public resource of the kind, both in terms of number of word pairs covered and languages considered. Additionally, the proposed method can be used to generate or detect pairs of false friends for any pair of languages, without requiring expensive manual work or dictionaries, but only large monolingual corpora to train word embeddings on, and small bilingual dictionaries to perform embedding space alignments.
The unsupervised nature of the algorithm proposed also has the advantage of a high coverage of the vocabulary, unlike dictionary-based methods, which are prone to becoming outdated as language evolves. One disadvantage of our embedding-based algorithm is the lack of distinction between different senses of the same word. In the future it would be interesting to continue the study in the direction of considering also context-specific senses of words, in order to be able to better handle partial false friends, which are pairs of cognates which share meaning in some contexts and not others. Along with false friends pairs, we publish a falseness score for each pair, which can be used to customize the sensitivity to difference in meaning that defines a pair of false friends according to the application. We believe this resource can be very valuable for language learners, for example by incorporating false friends pairs in a tool to aid with language acquisition or text comprehension for non-natives, as well as for machine translation or other applications using natural language processing in a multilingual setting.