{'fraction_non_alphanumeric': 0.03818975784041286, 'fraction_numerical': 0.010956728860658991, 'mean_word_length': 4.966840360018948, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Myanmar (Burmese)-English Named Entity Transliteration Dictionary

Transliteration is generally a phonetically based transcription across different writing systems. It is a crucial task for various downstream natural language processing applications. For the Myanmar (Burmese) language, robust automatic transliteration for borrowed English words is a challenging task because of the complex Myanmar writing system and the lack of data. In this study, we constructed a Myanmar-English named entity dictionary containing more than eighty thousand transliteration instances. The data have been released under a CC BY-NC-SA license. We evaluated the automatic transliteration performance using statistical and neural network-based approaches based on the prepared data. The neural network model outperformed the statistical model significantly in terms of the BLEU score on the character level. Different units used in the Myanmar script for processing were also compared and discussed.


Introduction
Transliteration is the task of transcribing words from a source script to a target script. Generally, the transcription is phonetically based. Transliteration processing is important for many downstream natural language processing (NLP) tasks, such as machine translation and information retrieval. In recent years, general sequence-to-sequence processing techniques for NLP tasks have been developed significantly. However, a lack of resources is still a problematic issue for many understudied languages. In this study, we focus on transliteration between Myanmar (Burmese) and English. To facilitate the application of data-driven approaches, we manually collected a dictionary containing more than eighty thousand Myanmar-English transliteration instances. The data have been released under a CC BY-NC-SA license for research purposes. 1 Based on the dictionary, we conducted experiments on automatic transliteration between Myanmar and English. Specifically, we conducted experiments using two neural network (NN)-based approaches: the Transformer model using the OpenNMT system 2 (Vaswani et al., 2017;Klein et al., 2017) and a joint agreement bidirectional long short-term memory (LSTM)-based recurrent NN (RNN) using the JANUS 3 tool . A traditional phrase-based statistical machine translation (PBSMT) system using the Moses 4 toolkit (Koehn et al., 2007) was set as a baseline. The experimental results were evaluated using the BLEU score (Papineni et al. 2002) on the character level. The experimental approaches performed well on transliteration tasks. The NN-based approaches outperformed the traditional PBSMT by large gains. The effect of using units at different granularities in the Myanmar script was also investigated. To the best of our knowledge, this study is the first systematic work on the topic of Myanmar-English transliteration driven by a relatively large-scale dataset.
The remainder of this paper is organized as follows: In Section 2, related work is described. In Section 3, issues for the transliteration of Myanmar are addressed and the collected dictionary is described. In Section 4, the experimental results are reported, and in Section 5, a discussion is provided. In Section 6, the paper is concluded and future work is presented.

Related Work
Many Asian languages apply special writing systems, and efforts have been made on transliteration processing for major languages such as Chinese, Japanese, and Korean (Merhav and Ash, 2018). However, studies are required on understudied languages with limited resources. Generally, the transliteration task can be modeled as a simplified translation task on the character/grapheme level rather than the word/phrase level, with no (or few) reordering operations. The technical background has been well established in the field of NLP. A PBSMT system (Koehn et al., 2003) can be used as an off-the-shelf tool once adequate data are provided. In recent years, NNbased frameworks such as the LSTM-RNN (Cho et al., 2014) have been widely applied to many NLP tasks. The Transformer model (Vaswani et al., 2017), which introduces a self-attention mechanism, is a state-of-the-art NN architecture in the NLP field. Regarding specific studies on transliteration, Finch et al. (2016) proposed an agreement model of bidirectional RNN, which outperformed PBSMT on various language pairs. Wu and Yarowsky (2018) compared several machine translation methods for the transliteration of 591 languages into English. Their conclusion was that a PBSMT system outperformed other systems including NN-based approaches. Regarding the case of Myanmar processing, there are few previous works. Ding et al. (2017) first attempted a Myanmar name Romanization task, where NN-based approaches did not outperform the traditional approaches of the conditional random field and support vector machine. It can be considered that the transliteration task may be sensitive to the quality and quantity of training data, in addition to the features of the specific languages involved, so proper approaches should be investigated case by case.

Overview
The Myanmar language belongs to the Sino-Tibetan language family. It is the official language of the Republic Union of Myanmar and the native language of the Bamar people. The Myanmar script is an abugida system. It consists of 33 characters of standalone consonants, four dependent consonants, and tens of diacritic marks that represent vowels and tones. The orthography of Myanmar is generally syllable-based, although syllables may be merged in special writing forms. One word can be composed of multiple syllables and one syllable can be composed of multiple characters. Between the characters and syllables, sub-syllable units may be identified for specific tasks. More details are available in Ding et al. (2017). The contemporary Myanmar language contains a considerable number of borrowed words from English. Temporarily, there is no consistent standard in Myanmar to transcribe the growing number of borrowed words from or through English.

Difficulty
The difficulty in modeling transliteration between Myanmar and English is two-fold. The first issue is the difference in the phonological inventory of the two languages. Chang (2013) provided a study on the phonology of English loanword adaptation in Myanmar. This study was based on a list of 280 collected borrowed English words. As typical features of the Myanmar consonant system, stops and affricates make a three-way contrast with voiced, voiceless, and voiceless aspirated; a two-way voicing contrast is also present with nasals and approximants. These contrasts do not completely exist in English. Myanmar has a typical seven-vowel system, and the syllable codas are restricted to nasal and glottal. Although the vowel system of Myanmar is somehow simpler than that of English, three tones in Myanmar are superfluous when transliterating into English. By contrast, consonant clusters are limited in Myanmar, but abundant in English. The second issue is non-phonetic orthographies. Both English and Myanmar apply an orthography that is more etymologically based than phonetically based. The Myanmar script has relative redundancy regarding its phonology inventory because many phonemes can be realized in more than one way. Moreover, some special spellings may be used intentionally to spell borrowed words for an exotic appearance. English spellings may also cause irregular transliteration where the transcription follows the spelling rather than the real pronunciation. As a result of the above-mentioned issues, transliteration between Myanmar and English becomes correspondingly complex. In the following are some examples. <eu> in "Eurasia" and <u> in "United Kingdom," although spelt differently, are transliterated identically as <ယူ > (/jù/) into Myanmar. This is also the case for some pronunciations that cannot be distinguished in Myanmar, for example, <aun> in "auntie" and <un> in "uncle" are both transcribed into <အန် > (/ʔàɴ/). By contrast, <j> in "Japan" is transcribed by pronunciation as <ဂျ> (/d ͡ ʑ.../); however, <j> is more traditionally and consistently transcribed as <ဇ> (/z.../) in the case of "January," "June," and "July."

Dictionary
To collect the transliteration instances, we began with two English-Myanmar parallel corpora, and then moved to resources on the Internet to enlarge the scale of the data. Specifically, we used the ALT corpus (Riza et al., 2016;Ding et al., 2018;Ding et al., 2019;Ding et al., 2020) 5 and UCSY corpus (Sin et al., 2018). 6 The ALT corpus consists of twenty thousand parallel sentences from news articles and the UCSY corpus contains two hundred thousand parallel sentences collected from different domains, including local news articles and textbooks. We used the GIZA++ toolkit 7 (Och and Ney, 2003) to obtain the raw alignments between the source and target language, based on which the transliteration instances were filtered. From the Internet, we further collected instances of places, organizations, and person names. The dictionary was encoded in Unicode. For experimental investigation, we divided it into three parts by taking the first one thousand lines as the test data and the final one thousand lines as the development data. Statistics for the data are listed in Table 1, where #Char. denotes Unicode characters, #Syl. denotes syllables, and #Sub-Syl. denotes the sub-syllable units proposed in Ding et al. (2017). Ta

Experiment
The Myanmar data were segmented into different units, as shown in Table 2, and all English data were lowercased. Moses was used to train the baseline PBSMT system. Character alignment was generated using GIZA++ and symmetrized using the grow-diag-final-and heuristics. A 5-gram language model on character was trained using the SRILM toolkit (Stolcke, 2002). 8 Minimum error rate training (Och, 2003) was used to tune the weight of the features. All other parameters were adapted using the default setting of Moses. Regarding the JANUS toolkit, which implements a bidirectional agreement LSTM-RNN model, the hyperparameters followed the original paper . Specifically, the settings were 500 for embedding, 500 for the hidden unit dimensions, and 16 for the batch size. AdaDelta was used as the optimizer, with a decay rate of 0.95 and ε of 10 -6 . The model was trained with 100 epochs. Regarding the Transformer model using the OpenNMT toolkit. The hyperparameters used in the experiments are listed in Table 3.

Discussion
In this section, we discuss case studies on transliteration instances that were difficult to process. In Tables 6 and 7, the outputs of different systems are compared for identical transliteration instances in both directions.  Table 6: Results for "cardiff" ↔ "ကာားဒစ် ဖ် ".

Unit
Some non-native Myanmar spellings may appear in the transcription of borrowed English words. Table 6 presents a typical example of the pair "cardiff" and "က ားဒစ် ဖ် ." The spelling of <စ် ဖ် > used to transcribe <iff> is not permitted in Myanmar native words. It can be observed that syllable-based processing cannot handle such exceptional structures, regardless of which system is used.  Table 7: Results for "djokovic" ↔ "ဂျ က ို ဗစ် ." Table 7 presents a difficult example of the pair "djokovic" and "ဂျ က ို ဗစ် ," for which no system provided the correct results for either the En→My or My→En direction. The word "djokovic" itself is not a native English word, so all systems provided the more common spelling of "jokovic" in My→En processing. <djo> also caused difficulty in En→My processing, and in most cases, <d> was transcribed separately as <ဒ...>. The most approximate result was provided by PBSMT using Myanmar syllables, where <dj> was correctly transcribed as <ဂျ...>; however, the vowel was not transcribed exactly. As the best performances of different Myanmar units do not differ so obviously in Tables 4 and 5, we consider that using character or sub-syllable units in Myanmar is a better option than syllables to avoid errors caused by irregular spellings, in most cases.

Conclusion and Future Work
In this study, we built and released a dictionary of Myanmar-English transliteration instances. Experiments were conducted on a baseline PBSMT system and two NNbased approaches to provide benchmark performance based on the prepared data. Temporarily, we are collecting more instances so that the scale of the dictionary exceeds one hundred thousand instances. We will investigate the effect of using the data to improve the performance of Myanmar-English translation in the near future.