{'fraction_non_alphanumeric': 0.0208078335373317, 'fraction_numerical': 0.0073439412484700125, 'mean_word_length': 5.244274809160306, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Chinese prosody phrase break prediction based on maximum entropy model

A maximum entropy based model for prosody phrase break prediction was proposed in this paper, and a comparison was conducted on large corpora between the new model and the decision tree based model which was the mainstream method for prosody phrase break prediction. The contribution of lexical information and influences of different cutoff values were also investigated. It was demonstrated that, utilizing the same information, maximum entropy based method made an improvement of 5.5% on F-Score over decision tree based method. Integrating lexical information, an improvement of 9.4% over decision tree was achieved. Using maximum entropy based method, to achieve the performance of traditional decision tree, 83% manual work could be saved.