{'fraction_non_alphanumeric': 0.03986206896551724, 'fraction_numerical': 0.01691954022988506, 'mean_word_length': 4.959178082191781, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
MASALA: Modelling and Analysing the Semantics of Adpositions in Linguistic Annotation of Hindi

We present a completed, publicly available corpus of annotated semantic relations of adpositions and case markers in Hindi. We used the multilingual SNACS annotation scheme, which has been applied to a variety of typologically diverse languages. Building on past work examining linguistic problems in SNACS annotation, we use language models to attempt automatic labelling of SNACS supersenses in Hindi and achieve results competitive with past work on English. We look towards upstream applications in semantic role labelling and extension to related languages such as Gujarati.


Introduction
Case markers express semantic roles, describing the relationship between the arguments they apply to and the action of a verb. Adpositions (prepositions, postpositions, and circumpositions) further express a range of semantic relations, including space, time, possession, properties, and comparison.
Languages have different strategies for encoding these kinds of semantic relations. Hindi-Urdu 1 uses a case-marking system along with a large postposition inventory (Kachru, 2006;Koul, 2008). Idiosyncratic bundling of case and adpositional relations poses problems in many natural language processing tasks for Hindi, such as machine translation (Ratnam et al. 2018, Jha 2017, Ramanathan et al. 2009, Rao et al. 1998) and semantic role labelling (Pal andSharma 2019, Gupta 2019). Many models for these tasks rely on humanannotated corpora for training data, such as the one created for the Hindi-Urdu PropBank (Bhatt et al., 2009), and in Kumar et al. (2019). The study of adposition and case semantics in corpora is also useful from a comparative/typological linguistic perspective, in comparing and categorizing the encoding of such relations across languages.
To that end, we release a completed Hindi corpus annotated for adposition and case semantic labels using the SNACS formalism (Schneider et al., 2018a. We approach the problem of automatic tagging of these labels using a variety of language models and explore what these models learn. Drawing on parallel SNACS corpora in English, German, Mandarin, and Korean, we compare strategies for encoding semantic roles across languages. 1 Hindi and Urdu are two registers written in two different scripts of a single language (usually called 'Hindi-Urdu' or 'Hindustani') with a largely identical grammar. While our corpus is in Hindi in the Devanagari script, the linguistic portions of our work (e.g. annotation guidelines) are applicable to Urdu as well.

Background
Hindi is a language of India, of the Indo-Aryan branch of the Indo-European family, and one of the bestresourced South Asian languages for research in natural language processing and computational linguistics (Joshi et al., 2020). Hindi has a small number of core case markers as well as a large class of adpositions for signalling semantic relations. We will discuss the linguistic features of case and adposition in Hindi below, related work from linguistics in this area, and introduce the SNACS schema.

Case and adposition in Hindi
Hindi is generally described as having three layers of case/adposition: the three basic morphological cases (example 1a), a small class of case markers/clitics that indicate core arguments to verbs (example 1b), and a larger class of postpositions governed by the genitive kā or ablative se (example 1c) (Kachru, 2006).
(1) a. bacce 'children ', baccoṁ 'children.OBL', bacco 'children.VOC' b. usne 'she.ERG', usko 'she.ACC/DAT' c. uske_liye 'for her', uske_nazdīk 'near her', uske_under 'under her' [code-switching] Masica (1993) grouped these three "layers" on the basis of historical development. Diachronically, morphological cases are the remnants of the Indo-European case system (via Sanskrit) that largely encode syntactic information, the case markers are Middle Indo-Aryan developments from spatial adverbs (e.g. Sanskrit upari 'above' > Hindi par 'LOC-on') that encode fundamental semantic roles on complements, and postpositions are more recent developments that even include borrowings from Persian, Arabic, and English and which indicate more concrete, e.g. spatial, relations between nominals. The case markers most commonly mark relations between verbs and their arguments and adjuncts, followed by relations between nominals. Case markers in Hindi are highly multi-functional even when using coarse descriptors from linguistic typology; e.g. se is described as indicating the ablative, instrumental, comitative, or comparative cases depending on context, respectively exemplified in (2).
(2) a. yahāṁ se jāō 'Go away from here.' b. cammac se k hā nā 'eating with a spoon' c. usse milūṁgā 'I will meet with him.' d. das se kam 'less than ten' That is not to say that se is three different case markers; the semantic role of a se-marked argument is just licensed by the predicate or other governor of the argument. Understanding how and in what context such markers indicate what semantic relations is an interesting problem. Thus far, there is no semanticallyannotated corpus of case and postposition semantics in Hindi, which motivated our annotation of this corpus.

Related work
There is a great deal of work on case and adpositions in Hindi. In syntax, some research topics are syntactic differences between morphological case, case markers, and adpositions (Spencer, 2005), the issue of differential case marking in the ergative and dativeaccusative (Bhatt and Anagnostopoulou, 1996;de Hoop and Narasimhan, 2005;de Hoop and Narasimhan, 2009;Montrul et al., 2015;Montaut, 2018), word order (Mohanan, 1994), and agreement (Montrul et al., 2012). On the other hand, there has been less research on the semantics of case and adpositions in Hindi. The mapping of case-marked arguments to lexical-semantic roles has been done in various computational projects (Begum et al., 2008;Vaidya et al., 2011). Paul et al. (2010) is an investigation of paraphrasing nominal compound relations with case in Hindi and English.

SNACS
The Semantic Network of Adposition and Case Supersenses (SNACS; Schneider et al., 2018aSchneider et al., , 2020) is a multilingual annotation scheme with 50 supersenses that characterize the use of adpositions and case markers at a coarse level of granularity. This scheme is akin to linguistic models of argument structure such as semantic roles and theta roles (including traditional categories such as AGENT and THEME), but expanded to include roles for adpositional relations, such as WHOLE for whole-part, SOCIALREL for interpersonal relations, etc.
A useful feature of SNACS is the construal system (Hwang et al., 2017), which allows an annotator to give one label for the morphosyntactic role or inherent lexical meaning (function) and another label for the predicatelicensed semantic relation (scene role) of a token. This is expressed as SCENEROLE↝FUNCTION if they differ. Examples of SNACS annotation for Hindi are given below. SNACS, thus far, has been used to annotate the English STREUSLE corpus (Schneider and Smith, 2015), The Little Prince in English and translations of it into Korean , Mandarin , and German (Prange and Schneider, 2021). There has also been annotation of L2 English (Kranzlein et al., 2020). This effort has been accompanied by the release of guidelines for annotator training, including for English  and Hindi-Urdu (Arora et al., 2021a). Some earlier works also discussed linguistic issues in Hindi annotation (Arora et al., 2021b).

Corpus and annotation
The corpus was the entirety of Nanhā Rājkumār, the Hindi translation of the The Little Prince by Antoine de Saint-Exupéry. 2 We used the SNACS annotation scheme, of which a brief overview is given in §2.3. Annotation was done by two Hindi speakers: A (the first author, who is a native speaker) and B (the second author, who is highly proficient) during June 2020-January 2021, and annotation guidelines were developed simultaneously (Arora et al., 2021a). Table 1 contains statistics about the final corpus, which was released in CoNLL-U-Lex format with Universal Dependencies annotations generated with Stanza (Qi et al., 2020).
There were two phases of annotation. In the first, A annotated the whole corpus (including all case markers and adpositions) and developed basic guidelines. In the second, B annotated chapter-by-chapter and A and B adjudicated disagreements concurrently. B also annotated focus markers, which were not included as targets in the first phase. A final pass was then conducted over the whole corpus to reconcile any remaining annotation disagreements.

Annotation targets
Following Masica's (1993) analysis of Indo-Aryan languages, we annotated the Layer II and III function markers in Hindi. These include all of the simple case markers 3 and all of the adpositions.
We also decided to annotate the suffix vālā when used in an adjectival sense (e.g. chot .ā -vālā kamrā 'the room that is small'), the comparison terms jaisā and jaise, the extent and similarity particle sā (chot .ā -sā kamrā 'small-ish room'), and the emphatic particles bhī, hī, to (Koul, 2008, 137-156). All of these modify the preceding token and mediate a semantic relation between their object and the object's governor, just as conventionally-designated postpositions do.
The directly-declined Layer I cases of nominative, oblique, and vocative were not annotated due to the much greater annotation load that would involve and how much greater the breadth of the annotations would be relative to other SNACS-annotated languages. This means verbal arguments without case clitics were not annotated. However, future work (especially with application to semantic role labelling) would benefit from such annotations, and similar work has been done on SNACS annotation of non-adpositionally-marked subjects and objects in English (Shalev et al., 2019).

Linguistic issues
Several linguistic features of Hindi-Urdu adposition and case semantics posed difficulties in annotating. Some are examined below. The annotation process itself relied on grammatical analyses of Hindi such as Koul (2008), dictionaries (McGregor, 1993;Dasa, 1965Dasa, -1975, and native speaker judgements. Functions for case markers Case markers encode little lexical content relative to adpositions. Table 2 shows the dominance of case markers in every category; given their versatility, delineating their prototypical functions is difficult. For example, a comparative in Hindi-Urdu is expressed with the ablative case marker se-should the function be SOURCE (as expected for the ablative case) or the narrower COMPARISONREF in this sense? This is an unresolved question; in labelling, we chose narrower functions when their use seemed to be a relation that is not completely supplied by the predicate.
In other cases, with highly polysemous markers such as se, it is difficult to pick a single function corresponding to an obvious grammatical case. For example, the verb pūchnā 'to ask' takes an argument, marked with se, indicating the person being asked. This instance of se could be construed as the ablative case (reflecting the return of a response from the person asked) or the comitative case (indicating a co-participant in communication, exactly as for verbs such as kahnā 'to say'). To resolve this issue we looked to typological evidence, in keeping with SNACS's multilingual aims: the closelyrelated language Punjabi, which has separate ablative (toṁ) and comitative (nāl) markers, uses the ablative in this construction, so we labelled the function SOURCE. Non-nominative/ergative subjects The AGENT is prototypically expressed with the ergative case marker ne or the unmarked nominative. To express modality, Hindi-Urdu, like other Indo-Aryan languages, employs various aspectual light verbs along with differential subject marking (de Hoop and Narasimhan, 2005). One example is the dative subject indicating obligation: In these, the subject's scene role is ORIGINATOR as it is a producer of writing. In example 7b, an expression of obligation, the subject is not only compelled to act by some outer force (fitting a THEME) but is also performing the action unaided (AGENT). SNACS currently cannot resolve the conflict between these two equally valid functions; we currently label example 7b as ORIGI-NATOR↝RECIPIENT in keeping with the morphosyntax of the dative subject. The issue is a broader problem of dealing with force dynamics in semantic role labelling, and may require new labels.
Other unconventional subjects are less problematic. South Asian languages near-universally have dative subject EXPERIENCERs (Verma and Mohanan, 1990). 4 For these, the prototypical RECIPIENT subject is fitting. The passive subject also has the unambiguous function of AGENT, just as the English passive by. Causative constructions Indo-Aryan languages, through suffixation, derive indirect and direct causative verbs from intransitive verbs. Indirect causatives take an argument in the instrumental case that is an impelled agent, grammatically distinguished from a true INSTRUMENT: Focus to (contrastive) 6.2 hī ("even") 3.6 bhī ("also") 3.0 Adpositions ke lie ("for") 3.3 ke pās ("near") 1.0 sā ("-ish") 1.0 kī tarah ("like") 1.0 jaise ("like") 1.0  Much like an obligated agent, the impelled agent takes part in two events, exhibiting properties of both AGENT and THEME. Furthermore, an impelled agent can control INSTRUMENTs of its own, and there cannot be two participants in the scene with the same semantic role (Begum and Sharma, 2010). For SNACS, Shalev et al. (2019)

mentioned similar issues in English.
This construction was rare in our corpus, but we find the best solution for this is a new label for animate and ambiguously volitional counterparts to INSTRUMENT in the SNACS hierarchy, much like the distinction between inanimate CAUSER and animate AGENT. Emphatic particles Following work on SNACS for Korean, which created a new label FOCUS for "postpositions that indicate the focus of a sentence (FOC), contributing information such as contrastiveness, likelihood, or value judgements" , we found that the Hindi emphatic particles hī 'only', bhī 'also, too', to (contrastive), and some uses of tak 'even' function as focus postpositions and thus merited annotation.

Corpus analysis
Annotator agreement 2,368 targets (79.7% of the total) were annotated independently by both annotators.  Table 3: Raw agreement on targets with at least 20 doubly-annotated instances in the corpus, sorted by agreement on the construal. Case markers are in bold.
In the first round of annotation by annotator A, the focus markers and a small number of case markers were not annotated.
Cohen's κ between both annotators for doubleannotated targets was 0.78 on scene roles, 0.85 on functions, and 0.73 on construals (role↝function), all of which are very high even compared to previous work on SNACS. It is not surprising that functions, which are inherent to the target type and less dependent on semantics, are easier to annotate than scene roles. Table 3 shows raw agreements on high-frequency targets, sorted by agreement on the construal label. Among the case markers, ne (ERG) and ko (ACC/DAT) are the easiest to annotate, which is unsurprising given that their usage is very consistent syntactically (subjects and objects/indirect objects, respectively). The 1.27 185 kī tarah ("like") 0.74 29 sā ("-ish") 0.47 31 ke bāre meṁ ("about") 0.00 23 bhī ("also") 0.00 90 hī ("even") 0.00 107 Table 4: Estimated entropy of targets with at least 20 instances in the corpus. Case markers are in bold.
low agreement on tak (ALL, "until, up to") was due to uncertainty over whether it indicates the endpoint of movement (GOAL) or the length of the distance covered to the endpoint (EXTENT); after adjudication, we standardised on the latter. The adposition ke pās "near" had a similar problem, where we disagreed on whether POSSESSOR was an inherent syntactic function of it or a semantic extension of its spatial use.
Marker and tag distributions Counts of targets and labels are presented in table 2, which shows that case markers generally indicate core arguments of verbs (e.g. AGENT as in subjects of verbs) and basic spatial relations (LOCUS, SOURCE), focus markers have discourse uses, and adpositions indicate non-core adjuncts (e.g. PURPOSE 'in order to'). Since Hindi has case markers, annotated targets were dominated by a few types with very large semantic breadth. We can operationalise a measure of semantic range using the entropy of the distribution of scene role labels, which are a coarse representation of semantics, for each case marker. Given a distribution x (the scene roles) with classes K, Shannon entropy (in bits) is defined as: We further adjust for the sample size and distribution using the entropy estimator due to Chao and Shen (2003), which is suited for linguistic distributions (Arora et al., 2022). In table 4, we report entropy of scene role for adpositions and case markers with at least 20 occurrences in the corpus. Case markers, as expected, occupy the top 5 places. However, tak ("until, up to") and ergative-case ne are much less semantically diverse than the other case markers. Some of the more frequent adpositions  are also very semantically diverse, but most are not and form a long tail.

Automatic tagging
Given the recent abundance of language models (both multi-and monolingual) for Hindi, we were interested in how well SNACS labels could be automatically tagged.
To that end, we trained a neural sequence tagger on the task of adposition and case marker segmentation and tagging of scene role and function. This is a subinstance of the lexical semantic recognition (LSR) task first proposed in Liu et al. (2021), who approached it with models similar to those used for named entity recognition (NER). Our tagger feeds the output of a contextual language model through a biLSTM then to a CRF which emits the final tagging. We loaded language models through HuggingFace (Wolf et al., 2020) and implemented our models with PyTorch (Paszke et al., 2019) and AllenNLP (Gardner et al., 2018).

Data preparation
In preparation for training a classifier, we converted the SNACS labels to the BIO scheme for sequence tagging. We only marked the label to be predicted on the B-tag of each sequence. In the case of the B-tagged-word being segmented into subwords by the language model being used, we labelled all non-initial subwords as I. For example, using multilingual BERT the phrase uske pīche 'behind them (sg.)' is tokenised into subwords and tagged for scene role labels as: _uske _pī cha e

B-Locus I I I
The sentences in the dataset are randomly split 80/10/10 between train/dev/test. Training occurs on the train set with period checks against the development set to measure convergence. Scores are reported on the test set.
The outputs from the language models are inputted to a 2-layer biLSTM with dropout of 0.3. Its output goes to a CRF, and the highest probability tags are outputted through Viterbi decoding. The number of epochs trained {30,60}, the learning rate {0.0001,0.0002,0.0005,0.001}, and LSTM layer size {64,128,256,512} are manually tuned hyperparameters. We did experiment with other architectures (e.g. RNNs, Transformers instead of the LSTM) but this was the best architecture we found.

Results
We report F1 scores on tagging in table 5. The best model is the Indic-Transformers BERT, with a distilled version (that is more efficient) coming in a close second. It is surprising that multilingual language models perform much worse than monolingual ones; IndicBERT, for example, is barely better than the baseline. These results are also competitive with F1 scores on English SNACS tagging, which bodes well for future work on multilingual SNACS given the complexity of the Hindi case marker system.
One issue that was prevalent across models was tokenisation errors involving the Devanagari script. The Indic-Transformers models droppped the vowel markers for u,ū, e, ai, the bindu (nasalisation marker), and the halant (vowel-killer) while tokenising. Somehow, they still were the highest-performing models; it is likely that with a fixed tokeniser and retraining they could have been even better.
Nevertheless, these are promising results, especially considering that the complex Hindi case system requires knowledge of verb frame semantics to accurately tag with SNACS.

Conclusion
We released an annotated corpus for Hindi of semantic relations encoded by case markers and adpositions, using the multilingual SNACS schema. We presented analysis of the distribution of labels and annotator agreement, explored linguistic issues encountered in annotation that pose problems for SNACS, and ran experiments on automatic sequence tagging for SNACS in Hindi with language models and biLSTM-CRF. We show that this is a feasible computational task and hope that this guides further work on SNACS for other languages, especially for those related to Hindi.
Future work on SNACS could consider multilingual comparisons, building upon work on aligning Korean and English annotations  and multilingual tagging as explored in this paper. Particularly, there is ongoing work on SNACS annotation of Gujarati, a langauge closely related to Hindi; multilingual tagging of the two would be an interesting next task. Leveraging SNACS annotations for upstream tasks is also underexplored, despite a growing interesting in the semantic relations encoded in prepositions which have otherwise been understudied in NLP (Elazar et al., 2021). We hope that this corpus will also be useful for future study on semantics-reliant tasks in Hindi.