{'fraction_non_alphanumeric': 0.02639840182648402, 'fraction_numerical': 0.004994292237442922, 'mean_word_length': 4.929357021996616, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Applying Statistical Post-Editing to English-to-Korean Rule-based Machine Translation System

Conventional rule-based machine translation system suffers from its weakness of fluency in the view of target language generation. In particular, when translating English spoken language to Korean, the fluency of translation result is as important as adequacy in the aspect of readability and understanding. This problem is more severe in language pairs such as English-Korean. It’s because English and Korean belong to different language family. So they have distinct characteristics. And this issue is very important factor which effects translation quality. This paper describes a statistical post-editing for improving the fluency of rule-based machine translation system. Through various experiments, we examined the effect of statistical postediting for FromTo-EK 1 system which is a kind of rule-based machine translation system, for spoken language. The experiments showed promising results for translating diverse English spoken language sentences. 1 FromTo-EK is an English-to-Korean rule-based machine translation system for some various domains (patent, paper, email and messenger).


Introduction
There have been many improvements in machine translation from rule-based machine translation (RBMT) to the latest statistical machine translation (SMT). Approaches for machine translation can be typically classified into conventional rule-based approach and statistical approach (Jin et al., 2008). RBMT translates a source sentence to a target sentence through analysis process, transfer process and generation process using analysis rules, dictionaries and transfer rules as its main translation knowledge. On the other hand, SMT system accomplishes translation using translation model and language model obtained from training large parallel corpus composed of source sentences and the corresponding target sentences (Koehn et al. 2003). Comparing two approaches, they have opposite features. That is, rule-based approach is better than statistical approach in the aspect of translation accuracy. However, fluency is contrary to each other. The language pairs that linguistic differences are huge such as English-Korean show these kinds of features apparently. We aim to improve the translation fluency of RBMT system by introducing SPE. The proposed method is similar to SMT. Difference is the composition of parallel corpus used to build statistical models. To build model for post-editing, parallel corpus should be ready, which is composed of the pairs of the sentence translated by RBMT and the corresponding correct sentence translated by human translators. Using this parallel corpus, we can build statistical model to post-edit RBMT results. Also, we explain some points to consider when applying SPE to English-to-Korean translation by various experiments. Our method consists of the following steps:  Constructing parallel corpus composed of translation results by English-to-Korean RBMT system and translation results by human translator of English source sentences.
 Building translation model and language model for applying SPE (at this phase, SMT toolkits are used).
 Applying decoder for SPE to the output of RBMT system.
The section 2 of this paper presents weakness of conventional rule-based machine translation system. And the overview of our method will be described in the section 3. The section 4 describes experimental parameter, experimental results. In the section 5 and the section 6 we sum up the discussion and show the future research direction.
2 Weakness of RBMT system Figure 1 shows the configuration of our rule-based machine translation system, FromTo-EK. The flow of machine translation as follows. First, roots of words in an input sentence is restored and part-ofspeech (POS) tagging is carried out by morphological analysis and tagging module. Second, syntactic structure is found out by syntactic analysis module (parser). FromTo-EK engine employs full parsing strategy to analyze English source sentences. Third, input sentence structure (parse tree) is transferred to adequate target language structure using transfer patterns. At this step, lexical transfer based on context is conducted using dictionaries and word sense disambiguation knowledge (Yang et al., 2010). Fourth, Korean generator generates final Korean translation sentence. The advantage of RBMT engine is that it can catch the exact dependency relation between the words in input sentence. It is very helpful to achieve high translation accuracy. In particular, in the case of language pairs which are very similar in the sense of linguistics (for example, Korean-Japanese), rule-based approach has showed good translation performance. However, the problem of RBMT system is its poor fluency compared to SMT system. In particular, in translating spoken language sentences, such features are outstanding. Actually, when translating English spoken language sentences, it is found that unnatural, rigid and dried expressions are frequently used. It results from stereotyped language transfer phase, translation knowledge and the limit of translation methodology. We cannot achieve the fluency like human translation by assembling translation knowledge pieces. This is why we propose SPE for RBMT system.

Target of post-editing
First of all, to find out the target of applying SPE, we manually evaluated FromTo-EK (English-to-Korean RBMT system) using test set composed of 200 English spoken language sentences. The measure for human evaluation is showed in Table 1 (Doyon et al., 1998). The score of one sentence in test set is the average of scores assigned by three human translators.   Table 2, analysis of the result of human evaluation shows that the sentences under average 2.5 have errors caused by analysis failure (including POS tagging errors and parsing errors). In this case, input to a transfer module (output from analysis module) is parse tree which already includes incorrect dependency relations. Because of syntactic (or morphological) analysis errors, these kinds of translation results have difficulty conveying the meaning of the original sentences. So, the sentences under 2.5 have little improvements by SPE. Meanwhile, in the case of the translation results scored over 2.5, it is not difficult to understand the overall meaning of source sentences except one or two words. Sentences over 2.5 have very little of errors from analysis phases. So, misunderstanding brought by analyzing source sentences incorrectly is rarely found. However, the sentences over 2.5 have some other problems. It may be summarized as Table 3. As we know by Table 3, the problem with the sentences over 2.5 can be divided into two categories. The first is intrinsic to English-to-Korean transfer module and Korean generation module. These are because of the limit of RBMT paradigm and the lack of translation knowledge. The second, on the other hand, is some different in that source sentence was well translated to target sentence without the loss of meaning. That is, the only problem is that in the aspect of the command of Korean, RBMT result is not so natural. At the same time, to improve the fluency of RBMT is the target of this paper.
factors comment Wrong position of adverb In Korean, the wrong position of adverb can change the object of modification.

Awkward expression
The lack of ambiguity resolution knowledge generates target words not matched in context.

Stereotyped target word
In most RBMT system, there is perfect strategy to select more natural target word depending on context in the aspect of target language. Table 3: Factors which cause fluency problems in FromTo-EK The fluency problem with RBMT system is that translation process has a mechanism with an emphasis on source sentence analysis. In RBMT approach, after input sentence analysis phase, transfer is based on just analysis result and translation knowledge such as dictionaries and transfer patterns. During source-to-target transfer and target sentence generation, the fluency of translation output sentence is not considered. In this paper, we demonstrate the impact of SPE for RBMT and explain improvements by SPE in the concrete.

Statistical Post-Editing Architecture
SPE is based on SMT. SPE and SMT differ from the composition of training corpus. For building translation model, SMT uses large parallel corpus composed of the pairs of source sentence and corresponding target sentence. The parallel corpus for SPE training is the pairs of translation sentence by RBMT system and the corresponding correct translation sentence by human translator.  We can build translation model and language model for SPE using this corpus composed of Korean (by RBMT) -Korean (by human). For building models for SPE, the tools for SMT are used at the same way. These models are used to post-edit RBMT results.  Figure 2 shows the configuration of SPE. We thought that by training based on alignment between incorrect machine translation result and correct human translation result, knowledge to improve the fluency of RBMT can be obtained. In applying SPE to RBMT system, SPE module is located at last place which take the translation result by RBMT as input. In figure 3, translation model and language model means statistical postediting models which are thought in the concept of statistical machine translation. We use the same tools as SMT for getting SPE models.  (Stolcke, 2002) version 1.5.12 for building a 3-gram language model Table 6: Baseline system for evaluation

Evaluation results
We tested the coverage of SPE using two different training corpora.  Table 7: The number of sentences changed by SPE Table 8 shows the analysis of the result of applying SPE to the result of FromTo-EK. In table 8, # sent, # imp and # deg means the number of sentences belonging to corresponding score, the number of sentences improved and degraded respectively. Table 8 presents something interesting. First, the characteristic of translation target language should be considered for better SPE performance. In this paper we focused English-to-Korean translation. Korean belonging to agglutinative language has distinct features that function words are well developed and these function words are closely related to fluency. So, when using parallel corpus composed of surface forms of words for training, SPE doesn't work well. Second, the target of SPE should be defined clearly. Because SPE doesn't consider source sentence in training phase, translation results under 2.5, including already analysis errors, don't have the advantage of SPE. This means that it is difficult for SPE to fix errors occurred at analysis phase. Third, translation results over 3.5 are apt to be degraded by SPE, contrary to expectation. This is related to reordering. We explain this with Table 9. In table 9, "word change" includes insertion, change and deletion. In FromTo-EK engine, function word is generated using just dictionaries and transfer patterns without consideration for rich context. This leads FromTo-EK into its own fluency problems, so does word of other part-ofspeech. However, we can know that these kinds of problems can be improved by SPE. In Korean spoken language, "subject omission" is frequent and SPE reflects it well. However, reordering raises serious side effects. Reordering frequently makes correct dependency relation incorrect. How to minimize the side effects of reordering is also important issue.
There are some examples of the result of applying SPE in Table 10. By SPE, we can see that word change including word insertion and word deletion occurred and fluency is improved. Final translation result by SPE is very good. The literary style expressions appeared in RBMT results are changed to colloquial style by SPE. And many abbreviation forms which are used in spoken language are introduced by SPE. Conventional RBMT cannot get such fluent sentences just using analysis rules, transfer patterns so on.

Discussion
We surveyed the effect of statistical post-editing method applied to English-to-Korean rule-based machine translation system. There still remain some problems to be solved necessarily:  How to select the sentences and words which could be improved by post-editing? It is needed to devise method to select the target to apply post-editing with minimizing degradation of translation quality. We are considering some language models for resolving these problems.
 Is there a method to apply post-editing for the part of sentence partially? If it is, how to find the phrase or clause to apply post-editing? This problem is similar to first problem to discuss. Up to now, table 9 shows word change category is best target of SPE. However, it is necessary to extend the boundaries of postediting.
 To improve performance, how to use other information such as source language information? Proposed method considers the alignment data only in target language aspects. If source language clues are used in combination with target language alignment data, the accuracy of post-editing will be improved.
 How to implement RBMT system with SPE module? It is the same as the structure which employs two translation engines sequentially. So its feasibility in application domains should be considered.

Conclusion
We proposed how to make use of SPE to improve the fluency of RBMT system. We could see that to maximize the effect of SPE, the consideration on target language has to be preceded. We categorized the factors which can increase the quality of RBMT by SPE. In the case of FromTo-EK system, a kind of RBMT, we knew that SPE works effectively for the sentences in which morphological or syntactic analysis errors did not occur. So, our next research topic is how to decide the target sentences/phrases/words of SPE. And, utilizing the source sentence and its related information (source word, its POS and its syntactic role) is also important research topic to effect SPE performance.