{'fraction_non_alphanumeric': 0.02028397565922921, 'fraction_numerical': 0.0, 'mean_word_length': 4.910179640718563, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Line-break prediction of hanmun text using recurrent neural networks

Recurrent neural network (RNN) has been broadly applied to natural language processing (NLP) and machine translation problems, by creating a deep learning model for sequential data. Hanmun is a Korean term for Chinese characters, and there are many cases in which Korean is pronounced by borrowing only the Chinese characters. Also, there are proper nouns and place names in the traditional Korean which are not used now. Therefore, we need a model for analyzing Hanmun rather than analyzing Chinese words in Chinese. In this paper, we propose a model for line-break prediction of Hanmun text using various types of RNNs. It is suitable for analyzing Hanmun meaning and usage vary according to the previous words. This model was used to segment the beginning and ending words of Hanmun characters and middle words. Experimental results show that our approach gets high performance in line-break prediction on Hanmun.