{'fraction_non_alphanumeric': 0.027224513847967, 'fraction_numerical': 0.005244549204478492, 'mean_word_length': 5.065403859899929, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Multi-Orthography Parallel Corpus of Yiddish Nouns

Yiddish is a low-resource language belonging to the Germanic language family and written using the Hebrew alphabet. As a language, Yiddish can be considered resource-poor as it lacks both public accessible corpora and a widely-used standard orthography, with various countries and organizations influencing the spellings speakers use. While existing corpora of Yiddish text do exist, they are often only written in a single, potentially non-standard orthography, with no parallel version with standard orthography available. In this work, we introduce the first multi-orthography parallel corpus of Yiddish nouns built by scraping word entries from Wiktionary. We also demonstrate how the corpus can be used to bootstrap a transliteration model using the Sequitur-G2P grapheme-to-phoneme conversion toolkit to map between various orthographies. Our trained system achieves error rates between 16.79% and 28.47% on the test set, depending on the orthographies considered. In addition to quantitative analysis, we also conduct qualitative error analysis of the trained system, concluding that non-phonetically spelled Hebrew words are the largest cause of error. We conclude with remarks regarding future work and release the corpus and associated code under a permissive license for the larger community to use.


Introduction
Yiddish is an example of a resourcepoor language, ex hibiting both general resourcescarcity and significant or thographic variation. In this paper, we present a multi orthography parallel Yiddish corpus based on Wiktionary. The corpus contains Yiddish nouns in several orthographic forms, and is intended to serve as a seed for further devel opment of transliteration and orthographic standardization systems. Our contributions are as follows: 1. We scrape all Yiddish nouns 1 , and generate a corpus containing parallel versions of each word in roman ized, YIVO, and diacriticless Chasidic orthography.
2. We bootstrap a transliteration model using Sequitur G2P (Bisani and Ney, 2008) 2 and obtain results that are sufficiently performant to be used in downstream tasks.
3. Finally, we release the corpus and code under a permis sive license. Our hope is that future research will use them to create larger, more unified language resources for Yiddish.

Orthographic Challenges of Yiddish
Yiddish has been traditionally been written using a modi fied version of the Hebrew alphabet (Jacobs, 2005). As a lowerresourced language, Yiddish presents several unique challenges to the development of language resources, and, ultimately, NLP systems.

Orthographic Variants
In terms of modern usage, there are three main orthographic variants of Yiddish. The first major variant is the standard ized Hebrew spellings developed by the YIVO Institute for 1 https://en.wiktionary.org/wiki/Category: Yiddish_nouns 2 https://github.com/sequitur-g2p/sequitur-g2p Jewish Research 3 , as well as their gold standard roman ized forms. The other prominent variants are the Hebrew spellings used by the Orthodox/Chasidic Jewish commu nity, which frequently appear in community publications. 4 For the most part, the YIVO orthographies have a oneto one correspondence with the actual pronunciation of the words, which enables relations between the Hebrew and romanized representations to be constructed with relative ease. A significant exception to this are words derived from Hebrew and Aramaic, which retain their original spellings when using the Hebrew alphabet. Compared to the YIVO standard, the "Chasidic" spellings tend to lack diacritics which are used in the YIVO stan dard to indicate vowels as well as the consonants b, v, f and p. Consequently, romanizing text written in the Chasidic orthography is much less straightforward than when using YIVO Hebrew spelling as sourceside data. Like the YIVO standard, the Chasidic system also retains the original non phonetic spellings of Hebrew and Aramaicderived word forms. Examples of words spelled in the various orthogra phies can be seen in Tables 1 and 2. Finally, there are also other less prominent orthographic variants, such as the spelling standard used in the Soviet Union (Erlich, 1973), characterized by the lack of word final character forms like ‫ך‬ and ‫,ף‬ as well as completely phonetic spelling of Hebrew/Aramaicderived words, e.g. ‫כאַסענע‬ instead of ‫.חתונה‬ However, as such spellings are nowadays largely obsolete in online Yiddish writing, they have been excluded from the present work.

Resource Availability
A large hindrance in the development of Yiddish language resources is the lack of suitable, sufficiently processed data sets. Firstly, a lot of resources, such as the digitized ebook  collection of the Yiddish Book Center 5 are not available in plain text but only as PDF files. Secondly, when plain text resources do exist, they are often written in a nonstandard orthography. For example, the Yiddish Wikipedia 6 , is writ ten largely in the Chasidic orthography, along with a small proportion of YIVO orthography mixed in. The use of non standard spellings makes romanization nontrivial, and cre ates problems for system development since the Unicode free romanized forms are often the simplest to work with in the context of building language technology systems. Fi nally, each collection of text typically only exists in one or thography, which further complicates the task of learning transliteration mappings due to a lack of parallel texts.

Related Work
For the most part, previous work on Yiddish NLP and cor pus creation has been scarce. Most efforts have attempted to address the resource availability bottleneck, and have fo cused on the creation of annotated corpora. Examples of such work include the aforementioned Digital Yiddish Li brary by the Yiddish Book Center, as well as the AHEYM speech corpus (Ćavar et al., 2016). There exists also The Penn Yiddish Corpus (Santorini, 1997), which contains ro manized Yiddish sentences along with POS tags and syn tactic parse trees. Unfortunately, the romanizations are largely ad hoc, and do not correspond to the YIVO standard.
In terms of orthographic standardization, Blum (2015) uti lized old Yiddish books and Optical Character Recognition outputs to transliterate nonstandard Yiddish spellings into standard YIVO spelling. However, it is not clear whether the associated models or corpora have been made publicly available.
On the machine translation front, Genzel et al. (2009) cre ated a EnglishYiddish and YiddishEnglish machine trans lation system based on a "convenience sample" of Yiddish data scraped off the internet, and postprocessed using var ious heuristics. While the performance of the translation system itself was promising given the resourcescarcity, less heuristic approaches are desirable to ensure generalizabil ity.
Finally, there has been some work on creating Yiddish word embeddings, as the language has been featured in research that has produced word embeddings for several languages 7 at once (Bojanowski et al., 2017; Grave et al., 2018. How ever, these approaches typically use Wikipedia as a source of training data, and as most of the Yiddish Wikipedia is based on the Chasidic orthography, the embeddings may perform poorly on downstream tasks if the new data uses a standardized orthography like YIVO. This further under scores the necessity to develop parallel corpora and tools for orthographic standardization, in order to ensure the ro bustness of downstream learned representations.

Data Scraping and Cleaning
We scrape all word forms from the Yiddish nouns cat egory of the English Wiktionary 8 using the lxml library in Python. Once the words have been scraped, we fil ter out rows that contain spaces, as those mostly cor respond to multiword expressions, such as idiomatic expressions.
After postprocessing, we obtain a fi nal corpus of 2750 word forms. The corpus is avail able for download at https://www.jonnesaleva.com/ multi-orthography-yiddish-corpus.

Filtering out NonStandard Spellings
Once the words containing spaces have been filtered out, we apply a rulebased system to detect potential nonstandard spellings among the words. The handcrafted rules are sim ple, and correspond roughly to detecting mismatches be tween the romanization and Hebrew alphabet form of the word. For instance, a word whose romanization contains ay but whose Hebrew script representation contains only ‫יי‬ instead of ַ ‫ײ‬ is flagged as potentially nonstandard. As the script identifies the potentially nonstandard word forms, the user is presented with the option of choosing one or more of the Hebrew spellings in the corpus to represent the YIVO spelling of the given word. Notably, this approach sometimes results in multiple spellings being chosen, as the same romanization can corre spond to multiple valid YIVO spellings, particularly when the romanized form also corresponds to a Hebrewderived word. For instance, the corpus contains two valid Hebrew alphabet spellings for oder: ‫דער‬ ‫אָ‬ and ‫דר‬ ‫,אָ‬ corresponding to or and the Hebrew month of Adar, respectively. Finally, to ensure that the correct YIVO spelling is recorded, the spellings of Hebrew/Aramaicderived terms whose spelling is nonphonetic are manually looked up in the Comprehensive YiddishEnglish Dictionary 9 (Beinfeld and Bochner, 2013). As can be seen in Table 2, the YIVO spelling and diacriticless Chasidic spelling of these terms is not always identical.

Dediacritization
After processing the corpus such that only word forms obey ing YIVO spelling rules are retained, we produce the Cha sidic orthography word forms by simply removing all dia critics from the YIVO spelling. In normal usage, such as on Yiddish Wikipedia, there is some writertowriter variation in the diacriticless spellings; however, we opt to remove all diacritics to produce the most challenging training data.

Experimental results
To demonstrate a potential use of the corpus, we train transliteration models to map between YIVO, Chasidic, and romanized orthographies. Specifically, we train models for three separate experimental conditions, corresponding to romanization, deromanization and diacritization. The scenario of mapping from YIVO to Chasidic orthographyor "dediacritization"-is trivial to implement deterministi cally by replacing diacritics in words with the empty string; therefore, a special model is not trained for it. We train our models using a popular graphemetophoneme conversion toolkit, SequiturG2P. It should be noted that while the performance of SequiturG2P is by no means stateoftheart, it is a good candidate for a baseline model as it works well "off the shelf" without requiring a lot of training data. All models are trained using the default set tings, except for supplying necessary flags to indicate UTF 8 encoding. As with the corpus creation code, we release the models and accompanying scripts under the MIT License 10 . Error rates are given in Table 3, along with the sizes of train and test sets (in words). All error counts were computed by SequiturG2P, with "string errors" referring to the number of words in which any error occurred, and "symbol errors" referring to the total number of errors encountered in the train/test set. Errors are defined as insertions, deletions or substitutions.

Romanization: YIVO → Romanized
The romanization performance of SequiturG2P is, by and large, impressive, and the model seems able to capture the regularities of the Yiddish writing system very well. This is the case for both Germanic words -e.g. ‫שמירקעז‬ is mapped correctly to shmirkez, and ‫רבעטער‬ ‫אַ‬ to arbeteras well as Slavic words, where ‫ג‬ ‫יראָ‬ ‫פּ‬ is mapped correctly to pirog. Errors made by the romanization model largely occur in the case of Hebrew/Aramaicderived words, as they are far less regular in pronunciation. The model tends to under insert vowels due to the lack of explicitly indicated vowels, 10 http://www.jonnesaleva.com/yi-lrec e.g. ‫רשה‬ ‫פּ‬ is mapped to prshe instead of the correct roman ization, parshe. The model also tends to insert incorrect vowels, such as predicting haskale instead of haskole for ‫לה‬ ‫ׂכּ‬ ‫.הש‬ Interestingly, this type of error mimics the errors made by beginning students of Yiddish. 11 A full set of sam ple romanization outputs can be seen in Table 4.

YIVOization: Romanized → YIVO
In terms of converting from romanized orthography to YIVO standard spelling -or, YIVOization -the model is again able to capture most of the regularities of Yiddish spelling. It successfully handles both Germanic words, such as geburt, mapping it to ‫,געבורט‬ as well as Slavic words like aparatshik which it transliterates as ‫טשיק‬ ‫ראַ‬ ‫אַ‬ ‫פּ‬ ‫.אַ‬ Interest ingly, the model is also able to capture regularities such as the sentenceinitial shtumer alef, ‫,א‬ which appears in case a word starts with a vowel other than e. Thus, words like ikh are successfully mapped to ‫איך‬ instead of ‫.יך‬ In terms of Hebraic words, the model appears to incorrectly apply the regular spelling rules it has learned to Hebrew words as well. Thus, for instance, eytse gets mapped to ‫אײצע‬ as opposed to the gold standard output, ‫.עצה‬ The model does pick up on some patterns between Hebraic words and their romanized spellings, such as the fact that an ending ye can correspond to ‫ה‬ instead of ‫.יע‬ Finally, as the model is purely statistical and has not re ceived any rulebased input about what sequences are valid Yiddish, it seems like the model often incorrectly uses the nonfinal forms of letters in a wordfinal position. As an ex ample, shif gets transliterated into ‫שיפֿ‬ instead of the correct form ‫.שיף‬ More examples can be seen in Table 5.

Diacritization: Chasidic → YIVO
While the diacritization model performs quite well, there are obvious drawbacks to its approach. A particularly se vere one is the tendency of the model to only predict the top hypothesis for any input, which implies that if several words have the same diacriticless representation, at most one of them can be diacritized correctly. This is can be seen in

Discussion and Further Work
Given the corpus and models outlined above, there are sev eral possible avenues for future research. As can be seen in Table 3, error rates tend to be rather high, particularly on unseen test data. While usually alarming, we feel that such overfitting is to be expected here given the small training corpus, and the fact that SequiturG2P has no information about character sequences that are likely in each orthogra phy a priori. As a potential avenue for future research, it would be useful to build more bespoke systems where prior knowledge about likely character transductions is explicitly incorporated into the model. In addition to domainbased prior knowledge, an overall transliteration system could benefit from unsupervised lan guage models trained on YIVO Hebrew and romanized spellings, which could act as regularizers, and contribute additional information in order to obtain a better estimate about the posterior probability of observing a predicted string given a source string. We feel that this could be par ticularly useful in transliterating the nonphonetic Hebraic words, and could reduce the amount of necessary training data for the model to perform well. Lastly, while the Sequiturbased models do capture a sig nificant part of the more regular components of Yiddish, overall it seems like they could benefit from Nbest out put and contextual reasoning to handle the more ambiguous cases, such as Hebrew and Aramaic spellings. If trained on a corpus of sentences, it is plausible that a featurebased approach where surrounding words are taken into account on the source side could substantially inform the translitera tion of a given focus word. This could be particularly useful in settings where a Chasidic spelling has multiple potential standardized spellings, but only one is correct given the sen tence context. Overall, it is our hope that the present work will inspire other researchers to develop further tools for Yiddish NLP.
Motivations for such research involve not only Yiddish scholarship per se, but also the prospect of building work ing NLP systems for languages that are lowerresourced and nonstandardized. Interesting ideas for future research include extending the present work to all Yiddish lemmas on Wiktionary, covering obsolete orthographies like So viet Yiddish, and building robust command line tools for transliteration. Such transliteration models can be used to normalize the orthography of existing corpora, such as the largely nonstandardized Yiddish Wikipedia. Once large standardized corpora become available, they can be used to further learn downstream representations such as word embeddings and build endtoend trainable models.

Acknowledgements
The author would like to thank Constantine Lignos for in valuable discussions and encouragement.