{'fraction_non_alphanumeric': 0.03488372093023256, 'fraction_numerical': 0.014338707376230561, 'mean_word_length': 4.708061889250814, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Propa-L: a semantic filtering service from a lexical network created using Games With A Purpose

This article presents Propa-L, a freely accessible Web service that allows to semantically filter a lexical network. The language resources behind the service are dynamic and created through Games With A Purpose. We show an example of application of this service: the generation of a list of keywords for parental filtering on the Web, but many others can be envisaged. Moreover, the propagation algorithm we present here can be applied to any lexical network, in any language.


Introduction
There are many applications that need lists of semanticallyrelated terms. For example, most parental control systems are based on so-called white (authorized) and black (unauthorized) lists. This raises two issues: (i) the cost of creation of these lists and (ii) the rapid obsolescence of the created lists. Thus, for parental control systems, according to (Cioffi et al., 2009): "it is really difficult to maintain updated the lists because of the high number of new resources daily introduced". This can be seen as a more general task of producing lists of semantically-related terms. This has been studied, in particular in (Habert et al., 1998), using syntactic similarities or similarity methods using WordNet (Fellbaum, 1998), as in (Varelas et al., 2005). However, these methods do not allow for the creation of dynamic resources (resources that evolve in time) and they involve resources that were costly to develop. The approach we propose allows for the creation of dynamic resources for a limited price. First, relying on crowdsourcing leads to the creation of lexical resources where terms associations and meanings can be updated almost in real time with real speakers usage. Second, term activation by propagation of information on a lexical network can be combined with filtering constraints. The higher the activation, the higher the probability that the term belong to the set defined by intention (constraints plus one or several seed terms). This type of approach allows for the computation of a term list that can be available on the spot. We first present the lexical network and the Games With A Purpose that originated the resource, then we detail the filtering service and show the obtained results on a parental control example.

JeuxDeMots, an Associative Game
At the heart of the system is the semantic network built through JeuxDeMots (Lafourcade, 2007), a Game With A Purpose (GWAP, see (von Ahn and Dabbish, 2008), (Siorpaes and Hepp, 2008) and (Thaler et al., 2011)) that has been on-line for seven years now, for its French version. Many lexical-semantic networks have been manually developed, like WordNet (Fellbaum, 1998), and its multilingual version (Vossen, 1998), or adapted as WOLF (Sagot and Fišer, 2008), among others, like HowNet (Lenat, 1995) and (Dong and Dong, 2006). Alternative similar resources have been developed by automatic extraction and crossing, like for instance BabelNet (Navigli and Ponzetto, 2012) from Wikipedia. In the case of JeuxDeMots, the resource is built through the gaming activities of a large number of players. The principle behind JeuxDeMots is quite straightforward: players are asked to enter ideas associated with a term chosen by the system (see figure 1). Players get points if they have answers in common with other players. The more original the answer, the more points are rewarded, but the higher the risk of having no intersection at all with others. Figure 1: JeuxDeMots: give ideas associated with "manger" (to eat). Figure 2 shows the summary (recap) of a game. The player scores points: honor points that count for the ranking and credit points that allow to gain some control on the game. Having some experience, players can get to play games on specific lexico-semantic relations, like is-a, hyponym, characteristic, location, agent, patient, etc. Terms in common for a specific game are added (or reinforced) in the lexical network. The mechanism behind the game induces that people performing well will help constructing a quality resource and players below average will have a negligible im-pact on the resource contents. A galaxy of pseudo games (the definition is given in section 2.2.) were created around JeuxDeMots, including mainly LikeIt 1 , AskIt 2 or ColorIt 3 , that add information to the network. SexIt is one of them. The created resource is freely available under a Creative Commons License BY-SA 2.0 FR. 4 Note that there is only one lexical resource that is created and enriched , while each game focuses on different modes of acquisition or specific information.

SexIt, an Opinion Game
SexIt 5 allows players to identify a term as being related to sex or not (see an example in figure 3, for "beach"). It is a pseudo game: there is no score, no ranking, no gain, only a poll-like comparison with other players. It is played only because the topic is fun: we would have had more difficulties collecting data on taxation, for example. This is an important limitation for language resource building using that kind of interaction. Collecting information about vocabulary related to sex can be necessary, for example to detect pornographic contents. However, such information is insufficient for discriminating pornography from courses in biology, reproduction or anatomy which, while not being pornographic, contain their share of sex-related terms. SexIt is built on the semantic network obtained from JeuxDeMots. It enriches the network by adding links to a Sex and a NoSex nodes. The links are weighted according to the number of players who chose the relationship. As of October 2013, SexIt allowed to process approximately 11,000 terms -from "peau" (skin) to "à la plancha" (plancha-grilled) or "Monica Lewinsky"-, and generated 52,000 hits. The main issue with SexIt is that the positive responses cover a large number of topics, including reproduction, anatomy, contraception, pornography, seduction, . . . If we want to use the resource in a parental filtering application, for example, we need to filter it out. We therefore developed a service, allowing both for the filtering of the network, according to various criteria, and for the selection of terms by signal propagation from a point of view.

Algorithm
A filter is a set of conditions that a term should verify. In the context of the lexical network, a condition is the existence of the specific relation between the tested term and another term. For example, the filter {x The propagation algorithm is presented below (see Algorithm 1) and consists first in choosing a starting term along with a set of filtered terms and then propagating a large number of signals along the lexical network. A signal sent from the starting term with a strength k, will randomly wander in the lexical network considering only filtered terms. The selection of the next term is done pseudo-randomly with a probability proportional to the strength of the relations. The stronger a relation, the higher the chance for it to be selected. Each time a signal reaches a term, the term value is increased by one, and the strength of the signal decreased by one. Hence, a signal with strength 3 can wander up to distance 3 from the starting term. A simplified view of the JeuxDeMots network, before and after propagation, is presented in figures 4 and 5, respectively.

Results
Propa-L is freely available online. 6 The interface provides the user with a number of parameters to fill in: • a starting term: this term is the focus from which other terms of the lexical network are going to be enumerated or skipped; Figure 4: An overly simplified view of the JeuxDeMots lexical network: words and word meanings are linked with typed and weighted relations. Any type of information take the form of a node linked to some terms. In the case of SexIt, two internal nodes, SEX-YES and SEX-NO, are pointed by terms when users are playing. This principle can be generalized to any type of information polling.
Data: A lexical network L, a starting term S ∈ L, a signal strength K, a number of signals n, a filter F . Result: A modified version of L where node weights are set. initialize all node weights to 0; while n > 0 do current_node ← S; k ← K; while k > 0 do current_node ← random_next_node(current_node); weight(current_node) ← weight(current_node) + 1; if not(current_node ≺ F) then k ← 0; else k ← k-1; n ← n-1; Algorithm 1: The propagation algorithm with filtering. If a term reached by the signal has been filtered out, then the signal is blocked. The selection of a next node is done pseudo randomly among the neighbors of the current node.
• a number of signals to be sent (cycles); • the strength of the signal; • a filter (can be empty).
The request produces a weighted list of terms. For example, with the starting term biologie (biology), 100,000 signals of strength 3 and a filter consisting in keeping only terms having at least one positive link in SexIt, we obtain the following list:  • spermatozoïde 1039 • bouche 1011 · · · It reaches far beyond SexIt, as the whole network can be filtered out using any constraint/filter. Increasing the strength of the signal leads to better recall but lower precision. Considering that the JeuxDeMots network is a small world, with a diameter of around 6, the best fit (optimizing F1-score between recall and precision) is 3 (half the diameter). The higher the number of signals, the more confident we can be in the relative order of terms in the list, but the longer the computing time (for 100,000 signals of strength 3, the computing time on our servers is around 15 seconds). When applied without any filter, with a starting term x, the result of the propagation algorithm converges toward the current state of the JeuxDeMots lexical network around x. The propagation algorithm can be alternatively used as a selector on constrained terms. For example, the following list has been obtained from the term botanique with the filter {x If the needed data is not available, it has to be produced, for example using a game like SexIt, LikeIt, or ColorIt.

Evaluation
The French JeuxDeMots network contains, as of October 2013, around 300,000 terms, including 15,000 to 20,000 word usages and more than 6 million relations. The main game has been played more than 1.3 million times by more than 3,500 registered players. For the SexIt data, more than 10,000 terms have been characterized as being related to sex (4,610) or not (8,158), for more than respectively 15,000 and 47,000 votes. There is, to our knowledge, no equivalent resource for French (the free French WordNet WOLF (Sagot and Fišer, 2008) is no complete yet), and the Princeton WordNet (Fellbaum, 1998), for English, is quite different, with a smaller coverage (155,287 words as in its version 3). Above all, none of the WordNet-like resources allow for the dynamic update JeuxDeMots offers. Furthermore, links between terms in JeuxDeMots are weighted. The higher the weight, the stronger the activation of the relation between the terms. Evaluating the JeuxDeMots resource is not easy, as there is no gold standard to compare it to. The same applies to SexIt. Indeed, apart from WOLF, there is no lexical resource available for French on which we can apply our approach. Moreover, even considering other languages, existing lexical networks do not provide the amount of information existing in JeuxDeMots, neither quantitatively nor qualitatively. Relevant information should include freely associated ideas, common sense knowledge and conceptual representations. These information are included in JeuxDeMots with SexIt (among others) as a result of the players activity. However, the principle behind the presented algorithm can be applied to any lexical network. It is however possible to manually evaluate the precision of the filtering service for some applications. There is no straightforward way to evaluate recall, again for lack of a gold standard. We therefore evaluated the performance of the filtering service on data filtered by the information collected from SexIt, as they are useful in applications like parental control, pornography contents filtering, etc. In the lists given above as example and those in appendix, we ask people if the terms they contain are relevant. More precisely, for the list above, they have been asked to count the proper terms that are related to sex under the focus of biology, and those which are irrelevant. The same task have been applied on terms related to sex under the focus of pornography. In the previous list (which is only partially given here), 98% of the terms were relevant (prison is the sole exception). However, some terms are clearly polysemous, and might be relevant to the constraints for only one of their possible meanings. One may ask, why not filtering terms from SexIt which have a majority of positive vote for sex (instead of having at least one vote)? The reason is that some terms, while being possibly related to sex, tend, in the eyes of people, not to be relevant in general, but could be, in a specific context (represented by the starting term).

Conclusion
We presented here a semantic filtering service applied on a semantic network and freely available via the Web. 7 This service is based on dynamic language resources, obtained by crowdsourcing using GWAPs. These results add to others in showing the fundamental shift in language resources building that is going on now (see (Chamberlain et al., 2013) for more details). If the use of crowdsourcing microworking platforms like Amazon Mechanical Turk can be questionable both in terms of ethics and of quality (Fort et al., 2011), GWAPs represent a change of paradigm that is only matched by the digitalization of resources. However, creating an efficient GWAP involves developments that are out of reach for many. This is the reason why we intend to develop a platform allowing users to generate their own pseudo-game from JDM.