{'fraction_non_alphanumeric': 0.02804642166344294, 'fraction_numerical': 0.0029013539651837525, 'mean_word_length': 5.197604790419161, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A bi-directional LSTM approach for polyphone disambiguation in Mandarin Chinese

Polyphone disambiguation in Mandarin Chinese aims to pick up the correct pronunciation from several candidates for a polyphonic character. It serves as an essential component in human language technologies such as text-to-speech synthesis. Since the pronunciation for most polyphonic characters can be easily decided from their contexts in the text, in this paper, we address the polyphone disambiguation problem as a sequential labeling task. Specifically, we propose to use bidirectional long short-term memory (BLSTM) neural network to encode both the past and future observations on the character sequence as its inputs and predict the pronunciations. We also empirically study the impacts of (1) modeling different length of contexts, (2) the number of BLSTM layers and (3) the granularity of part-o-speech (POS) tags as features. Our results show that using a deep BLSTM is able to achieve state-of-the-art performance in polyphone disambiguation.