{'fraction_non_alphanumeric': 0.02167487684729064, 'fraction_numerical': 0.013793103448275862, 'mean_word_length': 5.120481927710843, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Generating transcriptions for romanized Thai persons' names

A transcription of each word can either be produced by rules, statistical models, or retrieved from dictionary. However, the lack of standards and the variation of how a Thai person romanizes his or her name pose transcription a challenging task. Although the dictionary-based approach seems to produce the most accurate result, a letter-to-sound conversion module is necessary for unknown names. We propose an approach to transcribe romanized Thai person names into Thai sounds which considers the popularity of usage. The romanized Thai names are parsed into sequences of grams, utilizing the Gram lexicon, built from a corpus of more than 130,000 names. The results show 90 and 93% mean opinion score of acceptability when the transcriptions are generated from all possible sequences with unweighted and weighted Thai grams respectively. When longest-match model is used, the acceptability levels are 70 and 75% for unweighted and weighted Thai grams.