{'fraction_non_alphanumeric': 0.01866151866151866, 'fraction_numerical': 0.003861003861003861, 'mean_word_length': 5.170634920634921, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical Machine Translation of French and German into English Using IBM Model 2 Greedy Decoding

The job of a decoder in statistical machine translation is to find the most probable translation of a given sentence, as defined by a set of previously learned parameters. Because the search space of potential translations is essentially infinite, there is always a trade-off between accuracy and speed when designing a decoder. Germann et al. [4] recently presented a fast, greedy decoder that starts with an initial guess and then refines that guess through small “mutations” that produce more probable translations. The greedy decoder in [4] was designed to work with the IBM Model 4 translation model, which, while being a sophisticated model of the translation process, is also quite complex and therefore difficult to implement and fairly slow in training and decoding. We present modifications to the greedy decoder presented in [4] that allow it to work with the simpler and more efficient IBM Model 2. We have tested our modified decoder by having it translate equivalent French and German sentences into English, and we present the results and translation accuracies that we have obtained. Because we are interested in the relative effectiveness of our decoder in translating between different languages, we discuss the discrepancies between the results we obtained when performing French-to-English and Germanto-English translation, and we speculate on the factors inherent to these languages that may have contributed to these discrepancies.