{'fraction_non_alphanumeric': 0.029653877629615365, 'fraction_numerical': 0.0072690511721946755, 'mean_word_length': 4.56794425087108, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 25, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Introducing Lexical Masks: a New Representation of Lexical Entries for Better Evaluation and Exchange of Lexicons

The evaluation and exchange of large lexicon databases remains a challenge in many NLP applications. Despite the existence of commonly accepted standards for the format and the features used in a lexicon, there is still a lack of precise and interoperable specification requirements about how lexical entries of a particular language should look like, both in terms of the numbers of forms and in terms of features associated with these forms. This paper presents the notion of “lexical masks”, a powerful tool used to evaluate and exchange lexicon databases in many languages.


Introduction
Lexicon databases are at the core of many NLP systems. Yet their maintenance, evaluation and exchange between different systems can be cumbersome. Aside from the evaluation of the quality of individual entries, the evaluation of the consistency of the structure is a problem that is hard to tackle manually. From an interoperability perspective, several attempts have been made to homogenize the morphosyntactic features used to represent the lexicon, but few initiatives try to provide minimal requirements concerning the internal structure of a lexicon entry and its minimal specification. In this paper, we introduce the concept of lexical masks which aims at representing, in a consistent way, the expected internal structure of lexical entries. Masks are defined for each language and each part-of-speech in that language. We first describe the basic principles of lexical masks, how these masks have been designed for several languages, and how they have been used to automatically evaluate an existing lexicon. We also introduce different levels of specifications that account for the defectiveness of some lexical entries, but also allow some flexibility in the verification process. Finally, we describe how the masks are used in Wikidata to validate existing entries and help contributors to add entries more easily.

State of the Art
The interoperability of language resources has always been a challenge for the NLP community. The notion of interoperability covers two dimensions (Ide and Pustejovsky, 2010): syntactic interoperability and semantic interoperability. The first one involves the data format and communication protocol while the second one focuses on the meaningful interpretation of one system's data by another. In the domain of machine-readable lexicons, different projects have tackled the difficult issue of unifying features to describe lexical entries, allowing a meaningful semantic interoperability. Specifically, projects have aimed to unify the descriptors used to label specific forms in a lexical entry. Examples include the General Ontology of Linguistic Description GOLD 2010 1 (Farrar S., 2010) and the Universal Dependency Tree-bank 2 (Zeman et al., 2017) that inherently has to define a core set of tags to represent inflected forms. On the syntactic interoperability side, one of the most universally accepted models, the lemon-ontolex model, 3 describes the general structure of the lexicon (what is a lexical entry, how should it be organized, how to model and organize semantic, syntactic and morphological phenomena?). This model is largely inspired by previous models such as LIR Linguistic Information Repository (Montiel-Ponsoda et al., 2011), LMF Lexical Markup Framework (Francopoulo et al., 2006), or MILE Multilingual Isle Lexical entry (Atkins et al., 2002). More recently, OLiA Ontology of Linguistic Annotations (Chiarcos, 2012) aims at unifying the annotation terminology for linguistic phenomena (including GOLD). Thanks to these and similar projects, it is possible to define and reuse an annotation schema (and tag set) to describe lexical forms and features of the lexicon of any language. However, we are still lacking an approach that can specify how lexical entries should look like in a specific language, i.e. determining how many forms are expected and what features are necessary to describe these forms. This lack of model (together with an efficient way to represent it) makes it hard to evaluate the completeness and the coherence of lexical entries. The model also has to be operationalized in order to be automatically applicable in an evaluation process. In order to select a serialization to represent lexical masks, we chose ShEx (Prud'hommeaux et al., 2014) (see Section 5.), a language to describe shapes for RDF graph data. ShEx can be used to generate forms and to check data. There has been precedence by (Nielsen et al., 2019) for validating Danish Wikidata lexemes. We build upon this idea and propose to make the mask a standard representation for lexical entries in every language and for every part-of-speech. To this respect, the ShEx files are not only for validation (Section 4.) but also for creating input forms (Section 7.). The present paper proposes a solution for specifying lexical masks: it shows how we can, within the existing models, define the lexical entries in detail, and how we can use existing technologies to operationalize these definitions in order to check lexical entries. This solution also lends itself to be easily shared, and all masks that we create are being published to the commons, for shared ownership and maintenance.

Basic Principles
Lexical masks are specifications of the requirements a lexical entry should fulfill. In particular, a mask defines: • how many forms the entry should have to be complete; • what features are expected for each form.
Masks are specific to part-of-speech and language. One particular part-of-speech of one particular language can have more than one mask (see Section 3.3.).

SingNumber PlurNumber
MascGender form1 form2 FemGender form3 form4 For example, Table 1 shows the specification for Italian adjective entries. It specifies that four forms are expected, and each form should have one unique combination of gender and number features (i.e. there is one form for each feature bundle: MascGender / SingNumber, MascGender / PlurNumber, FemGender / SingNumber, and FemGender / PlurNumber). We do not commit to a specific tag set, but different tag sets can be used to represent the features. Of course, lexical entries can be (and often are) much more complex, both in terms of numbers of forms, but also in how the forms are being combined from the different available dimensions, in terms of the features used to describe these forms, and the entry in general. In the following, we review how our approach tackles this complexity.

Distinguishing entry-level and form-level features
Lexical entries are not only characterized by their forms and the features associated with the forms, but also by the feature assigned at the entry-level inherent to the entire entry. For example, the mask for Russian nouns (Table 2) shows an entry-level specification that requires the combination of animacy and gender features at the entry-level, and a set of form-level features, specifying that each form must have a case and a number feature. Examples for entry-level features include gender and animacy for nouns, aspect and transitivity for verbs, and degree for adjectives.

Accounting for more granularity: multiple masks
The configuration of lexical entries must also provide a certain level of flexibility to account for different structures of different entries. For example, we designed two masks for German nouns: the first mask, shown in Table 3 concerns nouns that have an intrinsic gender (i.e. at the entry level) and all the case and number declensions of that noun. The second mask, given in  3.4. Accounting for more granularity: canonical masks and "silver masks" In addition to allow multiple masks for a specific partof-speech, we also implement "silver masks" to account for phenomena such as defectiveness. Roughly speaking, defectiveness defines cases of incomplete morphological paradigms. When the defectiveness is regular enough, it should be specified in the lexical entry requirement. "Silver masks" define "smaller" entries and are a subset of the canonical mask, such as in the case of "weather verbs" in English (e.g. "it rains"), that only allow for a third person singular conjugation. Unlike multiple masks, "silver masks" should be used with caution, because they are only setup for a very small subset of lexical entries.

Implicit linguistic design
Designing lexical masks for a language requires us to take decisions on the structure of the lexical entries. Our model Table 4: German Nouns with Gender at the Form Level attempts to be consistent across languages, but necessarily implies theoretical choices. In particular, the distinction between "entry-level" and "form-level" features implies some decisions. For example, "degree" features for adjectives could be considered either as a form-level feature (and an adjective mask could have a duplicated set of forms for comparative and superlative forms), or as entry-level features.
Another example of a lexical design decision is for German nouns. As described above, we set up two different masks, one for 'standard nouns' with an inherent gender at the entry level, and one for nouns that have a gender inflection at the form level. Nouns denoting professions are the canonical example. Whereas standard text books of German grammar often don't mention nouns with gender given on the form-level, our experience shows that in many applications it is more useful to specify the gender at the form-level instead of the entry-level, linking masculine and feminine forms directly when they are morphologically marked. Note also that, since the nouns in Table 4 refer to descriptions for humans (professions, titles), they only support two genders (masculine and feminine), and is lacking a neuter gender. Similarly, paucal numbers in Russian can be seen as a pure syntactic construction influenced by the choice of numerals, or can be considered as a lexical features recorded as such in the lexicon (as expressed in the mask in Table 2).

Using masks for lexicon validation
As mentioned above, the evaluation of the internal structure of the lexicon (in term of consistency in the number of forms and the features used to described those forms) can be quite cumbersome. The mask model presented here is used to perform a semi-automatic evaluation of the lexicon we ingest in our database. As shown in Figure 1, each lexicon entry of a particular language (in the example an Italian adjectival entry) is ingested through the mask. During this process, we are checking that (1) this adjectival entry has indeed four forms, and (2) that each form has one of the required unique combinations of gender and number features (e.g. we cannot have two forms that are plural and feminine). This evaluation process will mark all the entries that are passing the masks as "structurally valid". The other entries that are not passing the masks will have to be looked at more carefully by a human rater. The interaction of masks with entries lends itself to a felicitous process of iterative refinement. It often happens that a bunch of entries are not passing our masks not because they are incomplete, but because our masks were set up only using the canonical representation of the lexical entry, but it Figure 1: Evaluating a lexical entry through a mask turned out that the entry is in fact more constrained (see Section 3.4.). For example, a number of French verbs were not passing our verbal mask. When looking at them, we realized that they were "intransitive verbs" that have only a single form for the past participle (and not the four expected forms, because they don't need to inflect). In such cases, we also create "silver masks" that are essentially subsets of the golden mask, and that "allow" the ingestion of valid entries, although not canonical. Such entries then need to be marked specifically as belonging to a more specific set, e.g. intransitive verbs, so that we can continue to select the most appropriate mask for validation.

Shared format: ShEx
The canonical data format of the Lemon-Ontolex ontology (McCrae et al., 2014) uses the RDF datamodel (Lassila et al., 1998). RDF is a very flexible model, but in the last few years it was increasingly recognized that in order to effectively use it, it is necessary to be able to validate the format for completeness and apply certain constraints before usage. This allows for the code using the RDF data to become much simpler. This lead to a number of proposals to describe and define constraints and completeness requirements for RDF data. The two most prominent languages currently in use for this task are SHACL (Knublauch and Kontokostas, 2017) and ShEx (Prud'hommeaux et al., 2014). In the following we will describe ShEx, as our use case uses ShEx (as described in Section 6.). The ShEx file in Figure 3 represents the mask from Table 3. 4 In line 12, the SPARQL (Pérez et al., 2006) query is given to find all lexicographic entries the ShEx file applies to (all possible focus nodes for the shape described by the ShEX file). Below then, we see the description of the lexical entry: in line 22, we require the grammatical gender to be given at the entry level, and in lines 23ff. we see the definition of the eight individual forms that constitute a German noun as per Table 3. The ShEx files can be used with any lexicographic data published in a data model that is compatible with the Lemon-Ontolex model. The ShEx files are all publicly available under a CC0 license (Creative Commons, 2009), and thus can be reused and modified as needed for the given use case. The validation will ensure that all required forms are present, that the right combination of grammatical features are given throughout the forms, and that all entry-level values are set as required. Furthermore, as usual with RDF, the validation will not prevent the data from having additional annotations and markers, e.g. it will not interfere with semantic annotations on the lexical entries, or linkages between entries from different languages. The ShEx files exclusively check for the completeness of the morphosyntactic forms of the lexical entry.

Use case: Validating entries in Wikidata
Wikidata (Vrandečić and Krötzsch, 2014) is a project by the Wikimedia Foundation and a sister project of Wikipedia and Wiktionary. It has the aim of creating a free knowledge base that anyone can edit. It started with the primary goal of supporting Wikipedia, and in particular it provides structured, linked data about anything that has a Wikipedia article (called a topic). In particular that means that there are already entries for many grammatical features, such as the different grammatical cases, numbers, genders, etc. In Wikidata, every topic receives a unique number, called Q Identifiers, which are language-independent. For example, the nominative case has the Q-ID Q131105 5 , the German language Q188, and the paucal number Q489410. Wikidata introduced the ability to add and work on lexicographic data in Wikidata in 2018. Since then, hundreds of thousands of lexical entries have been created. Wikidata publishes its data using the RDF data model. For the lexicographic data, Wikidata also follows the Lemon-Ontolex ontology, and therefore stays conceptually aligned with efforts to publicize lexicographic data on the linked open data Web (Chiarcos et al., 2013). In 2019, Wikidata started to use ShEx to define constraints and completeness requirements for its data . Although this feature was originally developed for the ontological data in Wikidata, it is also available for the lexicographic data. This allows us to enter and maintain ShEx shape files in Wikidata itself, where they can be collaboratively maintained and used to validate and check the completeness of Wikidata lexicographic data. This is the reason why we chose ShEx over SHACL to represent the constraints. In fact, we are not the first ones to follow this approach in Wikidata: (Nielsen et al., 2019) has previously Figure 2: A screenshot of Lucas Werkmeister's tool "Wikidata Lexeme Forms" for entering forms, available at https://tools.wmflabs.org/lexeme-forms created ShEx files for Danish lexicographic entries. We extend the method to generalize to more languages and use cases, with the goal to be useful well beyond the initial Wikidata use case. Wikidata is developing its platform and infrastructure to support ShEx files in a wide range of use cases across Wikidata. Most importantly for us, we can use the files we publish to validate existing lexicographic entries. This allows for the large semi-automatic validation of the crowdsourced entries in Wikidata, and thus provides a feedback loop for the community to see the quality of their entries. They can get a generated list of all entries that do not fulfill the constraints described in the ShEx files, and then decide case by case how to handle the data (i.e. whether it is a valid exception, whether it requires an alternative or silver mask, or whether the entry needs to be improved).

Possible usage: mask for UI
Besides using the lexical masks for validation, we also use them for a second use case: data entry editing.
Having a mask to enter the different forms and the required entry-level data can vastly improve not only the speed of entry and the quality of the entered data, but -and that might be most important -also the satisfaction of the contributors. Since Wikidata is a crowdsourced platform, the 1 PREFIX d c t : < h t t p : / / p u r l . o r g / dc / t e r m s /> 2 PREFIX o n t o l e x : < h t t p : / / www. w3 . o r g / n s / lemon / o n t o l e x #> 3 PREFIX wd : < h t t p : / / www. w i k i d a t a . o r g / e n t i t y /> 4 PREFIX wdt : < h t t p : / / www. w i k i d a t a . o r g / p r o p / d i r e c t /> 5 PREFIX w i k i b a s e : < h t t p : / / w i k i b a . s e / o n t o l o g y #> 6 7 # S t a n d a r d German noun 8 # German noun w i t h g e n d e r a t t h e e n t r y l e v e l and 9 # e i satisfaction of the contributors has a direct effect on how much data they enter over time, and on how long they remain active members of the Wikidata community. By encoding the possible forms and entry-level features beforehand, we significantly reduce the contributors' mental load while adding data. They don't have to switch between focusing on a lexeme and the actual individual forms, and the possible forms a lexeme in that language and part-ofspeech might have. They don't need to create the different forms and their respective features, but merely enter the actual forms into a form-based user interface. Note that this does not preclude exceptions. They still can be represented in the datamodel and entered through Wikidata's traditional user interface. In Wikidata, these exceptions can then be explicitly marked, e.g. as an intransitive verb or a plurale tantum, etc. The editing forms are created automatically from the released ShEx files, using existing Wikidata infrastructure developed for form-based entry of lexicographic data (see Figure 2).

Conclusion and Future Plans
In this paper we presented lexical masks as a mechanism to share specifications of lexical entries. We showed how we can use new, but existing technologies such as ShEx to easily apply the specifications of the lexical masks to a large lexicon with small effort, in order to validate the lexical entries, and find exceptions, incomplete entries, and areas where more work is needed. We have published the ShEx files for different parts-ofspeech and languages in Wikidata. We hope that by donating these to the public domain, we will maintain and extend them together with the wider lexicographic community. 6 We are also working on a process where we can develop more masks for languages we do not have expertise in with the wider community. We would offer the skills needed to formalize the masks as ShEx files, and work with contributors who have language expertise in creating the lexical masks. We hope that this way we can formalize a wider set of languages than has been available so far, thus expanding the reach of these technologies to many more languages. On a more technical side, we also plan to extend the mask model with more internal checks. For example, we'd like to include simple morphological patterns inside the forms of the mask, in order to account for generic morphological paradigms. This will also allow to automatically generate regular inflected forms, speeding up even more the editing process.

Acknowledgement
The first implementation of the mask infrastructure was done by AdnanÖztürel. We thank Lydia Pintscher and Lucas Werkmeister of Wikimedia Deutschland for their support.