{'fraction_non_alphanumeric': 0.02999501361666219, 'fraction_numerical': 0.01054811859920985, 'mean_word_length': 4.837886251679355, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
What Influences the Features of Post-editese? A Preliminary Study

While a number of studies have shown evidence of translationese phenomena, that is, statistical differences between original texts and translated texts (Gellerstam, 1986), results of studies searching for translationese features in postedited texts (what has been called ”posteditese” (Daems et al., 2017)) have presented mixed results. This paper reports a preliminary study aimed at identifying the presence of post-editese features in machine-translated post-edited texts and at understanding how they differ from translationese features. We test the influence of factors such as post-editing (PE) levels (full vs. light), translation proficiency (professionals vs. students) and text domain (news vs. literary). Results show evidence of post-editese features, especially in light PE texts and in certain domains.


Introduction
Over the past three decades, differences between translations and original texts have been substantially debated and empirically studied. Overall, research on these differences has shown that translations are usually normalised to conform to the linguistic norms and cultural aspects of the target language (Kenny, 2001). It has also been shown that translations tend to present less varied vocabulary (lower type/token ratio) and lower information load than original texts (Johansson, 1995;Laviosa, 1998). Statistical differences observed between originals and translations have been named translationese (Gellerstam, 1986;Baker, 1993;Volan-sky et al., 2013;Daems et al., 2017;Toral, 2019). According to Volansky et al. (2013), translationese phenomena are the result of two coexisting forces with which translators have to cope during the translation process: on one hand, fidelity to the source text and, on the other hand, fluency in the target language.
The term "translationese" had been put forward by Gellerstam (1986), but it was Baker (1993Baker ( , 1996 who proposed and described the linguistic and stylistic natures of translationese, naming them Translation Universals. Translation Universals are hypotheses of linguistic features common to all translated texts regardless of the source and target languages. The hypothetical features proposed by Baker are: Simplification, Explicitation, Normalisation (or Conservatism) and Levelling out (or Convergence, as named by Pastor et al. (2008). 1 Simplification means that translated texts are easier to understand than original texts because translators tend to simplify the language of the original text for the readers. Explicitation is the tendency to spell things out in translation; consequently, translations tend to be longer than original texts. Moreover, linguistic features that are typical of the source language are more explicit on the surface of the translation even though they are optional. Levelling out or Convergence means that there is less variation among translated texts than among non-translated texts. In other words, translated texts seem to be more similar to each other than original texts (Baker, 1996).
Within translation and machine translation (MT) literature, a number of studies (Baroni and Bernardini, 2006;Pastor et al., 2008;Volansky et al., 2013;Rabinovich and Wintner, 2015) have shown that computers can distinguish to a high degree of accuracy between translations and original texts. On the other hand, Daems et al. (2017) found that computers are not capable of accurately distinguishing between human translation (HT) and post-editing (PE), that is, the authors did not find any indication of post-editese in HT and PE texts. However, Toral (2019) has shown strong evidence that there is such a distinction. He found that PE texts contain post-editese features since they represent more interference from the original text than translationese features. A similar pattern was found byČulo and Nitzke (2016) who compared MT, PE and HT in terms of terminology and found that the way terminology is used in PE texts is closer to MT than to HT and it has less variation than HT. The study carried out by Vanmassenhove et al. (2019) also found evidence of post-editese features. In this study, the researchers compared MT and HT and found that current MT system processes cause a general loss in terms of lexical diversity and richness when compared to humangenerated translations.
Although evidence for post-editese has been reported in the literature, current results do not point to a clear conclusion. For example, in the studies previously mentioned, no distinction has been made between the levels of translation proficiency. In addition, the convergence feature has not been tested. In the present study, we fill in this gap. We base our experiments on studies by Toral (2019) and Daems et al. (2017) addressing post-editese. We aim to investigate the features that distinguish translationese from post-editese, that is, the unique characteristics of a post-edited text that set it apart from a translated text and an original text. We will simultaneously test whether linguistic patterns present in PE texts change as a function of three factors: the proficiency level of the translators (professional translators vs. student translators), text domain (news domain vs. literary domain) and PE type (full PE vs. light PE).
In the next section, we present our methodology in detail describing all the features investigated. The results are presented and discussed in Section 3. In Section 4, we present our conclusions with suggestions for future research.

Methodology
This section describes the corpora used for the experiment, the PE process, and the features we consider to verify the existence of translationese and post-editese in both human translation and the post-edited text versions.

Study Rationale
The rationale behind our experiments is the following: we will look for typical features in both HT and PE texts. If differences in feature patterns between those translation types are observed, then we assume that our corpus presents evidence of post-editese features. If, on the other hand, no differences between HT and PE texts are found, we assume that PE and HT are not distinguishable, as show by Daems et al. (2017). The research questions that guided our experiments are: • RQ1-Is it possible to find translationese features in PE texts?
• RQ2-If RQ1 is true, are there differences between the features extracted from PE texts and HT texts?
• RQ3-If RQ1 is true, do the domains of the texts, the proficiency of the translators and the type of PE influence the features analysed? If so, how?

Corpus
Two corpora were used for this experiment: the New York Times (NYT) and the Opus corpus. The NYT corpus is a collection of English headlines from The New York Times online newspaper, human-translated into Brazilian Portuguese (PT-BR) (Antiqueira et al., 2002). The NYT corpus consists of nine different texts 2 about general news. Because the corpus was not aligned, we decided to align it manually as some English source sentences were split into more sentences in PT-BR during the translation (one to many). The alignment was carried out looking into the source sentence and aligning all the correspondents in PT-BR in the same line. 3 In total, eight texts from the NYT corpus were used, with 251 sentences, and 6097 tokens in the source. From the Opus corpus (Tiedemann, 2012), we used a part of the subsection Opus Book 4 EN-PT. In total, 250 in-context sentences from Alice in Wonderland were used, with 5920 tokens in the source. In total, nine texts were post-edited by the translators: eight from NYT corpus (news) and one from the OPUS corpus (literature).

Translators, Tools and Guidelines
The corpus was translated using Google translate. 5 Four translators -two students and two professional translators -post-edited the corpus on two PE levels: light post-editing and full post-editing. Light PE was performed by one professional translator (PL) and by one student (SL), while full PE was performed by the second professional translator (PF), and by the second student (SF). Translators were given specific guidelines and were asked to follow them thoroughly. The tool used for the PE task was the PET tool (Aziz et al., 2012), and no time constraints were set for the task. A warmup task for the translators to get acquainted with the tool and guidelines was set up. Translators were encouraged to ask questions about the tool and/or guidelines if needed.

Features
A set of linguistic features were extracted from our corpus with the purpose of identifying the existence of post-editese as well as to test the effect of translation domains, translation proficiency and PE levels on the features analysed. The features examined are listed below.
Simplification -According to Baker (1993), simplification can be determined by comparing the vocabulary range and information load of the translated and original texts. As translators tend to split long sentences into smaller ones to facilitate text comprehension, simplification can also be reflected by number of sentences and sentence length. In the present study, simplification is computed by calculating lexical density (content words/words ratio), lexical richness (type/token ration), as well as sentence count and mean sentence length. (Daems et al., 2017;Toral, 2019).
Explicitation -Because translated texts tend to be more explicit than originals, they tend to be longer than original texts. Moreover, translated texts tend to follow the original in using pronouns even when they are optional in the target language (Volansky et al., 2013). This is the case of the language pair studied here: English does not allow subject omission, while for PT-BR an explicit subject is optional as tense, person and number information expressed by the subject can also be inferred from the structure of the verbs (Chomsky, 1993). In order to investigate explicitation phenomena, we test whether translations are longer than originals (length ratio), and whether the amount of personal pronouns (personal pronoun ratio) is different between translations and original texts.
Convergence -Translated texts tend to be more similar to each other than non-translated texts (Baker, 1993(Baker, , 1996Pastor et al., 2008). Convergence can be computed by calculating the variance of the features extracted within the original texts and within the translated texts (HT and PEs).

Results and Discussion
A series of ad hoc programs was written in the Python programming language to extract the linguistic patterns from the corpus and to identify the features. Descriptive statistical analysis was carried out in Language R. The automatic metric (h)TER was calculated using MulteEval. 6 Statistical significance was not calculated as we are aware that the size of the corpus and the number of participants is relatively small. It is noteworthy to mention that boxplots are presented to illustrate only the striking differences.

Automatic Metrics
We compute (h)TER (Snover et al., 2006) scores to measure the distance between the MT output against the HT, and the distance between the MT output and the PE versions. The higher the score, the more different MT is from HT, and the PEs from the MT. Table 1 shows the overall results for the automatic metrics for each translation type, while Figure 1 shows the results per domain. In both Figure 1 and Table 1, the first column/bar shows the scores for MT against HT, and the following 6 https://github.com/jhclark/multeval We observe that the MT is indeed quite different from the HT version (51.70), and that all the light PE versions were indeed lightly post-edited with both professionals and students reaching <2 in terms of hTER in average. Interestingly, we can see that there is more PE being performed in the literature domain than in the news domain ( Figure  1), where PF reaches 49 in terms of hTER against 21 in the news domains, and SF stands at 15 in literature against 8 in news domain, evidencing a domain effect on the amount of edits.

Simplification
Lexical Richness (LR) -In order to measure how varied the vocabulary range of original and translated texts is, we calculated type-token ratio (TTR), which is the number of token types, divided by the number of total tokens. We hypothesise that original texts will present higher lexical richness than the HT, MT and PEs versions. Because literature domain may involve more verbal artistry (e.g. paraphrase of figurative language and metaphors in the target language) (Baker, 1992), we hypothesise that the difference between originals and translation versions will be lower in the literature domain. Table 2 shows the overall results while Figure 2 illustrates the results per domain.
As previously mentioned, the literature on translationese shows that translated texts tend to be less lexically varied than original texts. When looking at the results per domain (Figure 2), we confirm our initial hypothesis. In the news domain, the original texts present higher lexical richness than the HT, while the MT version is very close to the PEs and the originals.   In the literature domain, the difference between originals and translated texts is more notable, where all the translation types present more lexical variety than the original. More interestingly, HT and the full PE versions seem to have less lexical variety than the MT and the light PE versions. We assume that this reverse pattern in lexical richness for literature could be due to two main reasons, one of a linguistic and the other one of a stylistic nature. As PT-BR contains more verbal forms than English, these forms increased the number of types per verb root. We found, for instance, 128 occurrences of auxiliary verbs in the HT version, but only 38 in the original texts. Thus, we as-  In spite of the unexpected results per domain, a pattern holds in both data sets: Because professional translators tend to pull the vocabulary range down in order to simplify text reading and comprehension and avoid redundancy, HT and PF versions tend to be similar. MT, on the other hand, tend to be closer to the original as observed by Toral (2019), and light PE, either professional or student, tends to keep the MT pattern.
It seems that this simplification feature is present in all translation types in the news domain, but its manifestation is more evident in translation types involving more human interference, namely: HT, PF, SF. Moreover, these results show a visible effect from domain, the proficiency levels of the translators.
Lexical Density (LD) -To measure the amount of information present in the original text and in the translated texts, we extracted LD features by calculating the ratio of the number of content words (nouns, verbs, adjectives, adverbs) to total number of words. In this experiment, we excluded auxiliary verbs. As lower lexical density is a way of building redundancy and making a text simpler, we hypothesise that HT texts, PF and SF present lower lexical density than originals, but the pattern for MT, PL and SL will be similar. Table 3 displays the overall results and Figure 3 illustrates the results per domain.
In the news domain, our results confirm our hypothesis, as original texts show a higher lexical density than the HT texts. The MT texts show higher lexical density than the HT and is followed by the PE versions. We hypothesise that lexical density is higher for MT texts because this translation type tends to be close to the original texts than HT texts, corroborating teh results of Toral Regarding the literature domain, our results contradict our hypothesis. The original texts present lower lexical density than the HT versions. As noted by the analysis of lexical richness, literary texts may feature more varied vocabulary of either lexical and function words as a way to conform to the linguistic norms and cultural aspects of the target language. In the literature domain, the PF version is equal to the HT, suggesting the number of edits performed has an effect on the translationese features. Interestingly, in both the literary and news domains, we confirm that MT and PEs present similar patterns.
Sentence Count (SC) and Sentence Length (SL) are calculated by simply counting the total number of sentences and the mean sentence length (in words). As mentioned previously, because translations tend to be simplified, we expect them to have a higher number of sentences and that those sentences will be shorter than the sentences in the original texts. Table 4 shows that original texts present, on average, slightly fewer sentences than the other translation types. In the news domain, the MT version presents a lower sentence count than the HT. Also, PE versions are closer to the MT than to the HT version. In the literature domain, no strong pattern can be observed, but light PE versions (PL and SL) tend to reduce the number of sentences compared to the full PE versions (PF and SF).   Regarding mean sentence length (Table 5, the original texts present slightly longer sentences on average. The MT version tends to follow the same sentence length of the original, and PE versions tend to keep the same pattern of the MT. Together, these results show that original texts tend to contain fewer sentences on average than HT texts as predicted, but PE versions tend to keep MT patterns, especially for light PE.

Explicitation
Length Ratio (LgtR) -According to Baker (1993), translated texts tend to be longer than originals. We test this hypothesis by calculating the difference between the length of the original text (measured in characters) and the length of the translated versions, divided by the length of the original. We expect translated texts to be longer than original texts. In Table 6, we observe that, overall, HT is 5% longer than the original, while the MT is even longer with 8%. The PE versions are closer to the HT than the MT version.
As predicted, overall results as well as results per domain confirm that translations are longer than the original. In Figure 4 and Table 6, we note that the literature domain does not, on average, differ in length from the original text. In the news domain, on average, the differences in length are more accentuated. MT texts present a greater variation (0 to -0.25), having its median further from the original, that is, further from 0.  Theses results suggest a domain effect and a posteditese effect since PE versions tend to be closer to the original texts in terms of length than HT versions. It is noteworthy, however, that differences in text length between originals and translation could be explained by English being a more concise language than Portuguese, not necessarily by the presence of explicitation. Therefore, in order to obtain a better picture of the explicitation phenomena, we tested if elements that are optional in the target language, such as the personal pronouns, were kept in the translations.

Figure 4: Length Ratio per domain
Personal Pronoun Ratio (PPR) -To test if translated texts tend to follow the original in using pronouns even when they are optional in the target language, we calculated the difference in the number of personal pronouns (PP) between original and translated text. While we expect the original texts to have a higher number of personal pronouns since they are optional in Portuguese, we expect that the MT version will be closer to the original than the HT, and that the full PE versions will be closer to the HT. Table 7 shows that, indeed, the original presents a higher ratio for PPR, given the positive ratio for all the translation types. Overall, the MT version is closer to the original (0.55) than the HT (0.59). While the PL keeps the  same ratio as MT, both student versions (SF and SL) slightly increase the ratio, revealing a PE effect and a professional proficiency effect.

Convergence
According to Baker (1993), translated texts tend to be more similar to each other than to the original texts. To investigate this hypothesis, we compare the variance scores obtained for the set of original texts, translated texts and post-edited texts for each of the simplification and explicitation features extracted from our corpus (literary domain and news domain): mean sentence length (MSL), sentence count ratio (SCR), lexical richness (LR), lexical density (LD), length ratio (LgtR) and personal pronoun ratio (PPR). For this comparison, we opted to calculate the variance within the translated texts involving only human translation (original and HT) separately from translations involving a MT (MT, PF, PL, SF, SL) to test the hypothesis that variance within MT and PEs is not as high as the previous experiments have shown, as well as to verify whether variance within the PE texts is higher or lower than the variance within the set of MT + PE texts.
Overall, Table 8 shows that original texts vary more than all translated texts for all features (MSL, SC, LR, LD), suggesting that the original texts are les similar to each other, while the trans-lated texts are more similar to each other. When comparing variance between the set of HT and the set of other translation types (MT+PEs and PEs), it is possible to observe that the variance scores are very close to each other for all translationese features, suggesting that translation type has little effect. Variance scores obtained for the MT + PEs set do not differ from the PEs set. This indicates that MT texts are very close to PEs texts in all features, except for a tiny difference in variance score obtained for the LgthR feature between the MT + PEs set and PEs text set.

Conclusion
This study investigated the presence of posteditese features in a corpus composed by HT, MT and PE texts post-edited by either professional translators or student translators in two domains: news and literature. Our results have revealed translationese features on the surface of HT and also PE texts, answering in the affirmative RQ1. Most of the features described by Baker (1993) were confirmed in the news domain for both HT versions and MT versions, namely LR, LD, LgthR, PPR, SC, except for mean sentence length. In the literature domain, not all translationese features were confirmed and, thus, we can assume that text domain plays a role in the prevalence of translationese features. This finding suggests that, looking for translationese features exactly as described by Baker (1993), may lead to erroneous conclusions, especially in the literary domain. This domain contain certain stylistic features that reflects the translator's verbal artistry, and issues inherently related to the language combination.
The most important finding of our study is the difference observed in the manifestation of translationese features between HT and PE texts, thus revealing evidences of post-editese features. Post-editese features were found to be reflected as more interference from the original than HT texts and also more interference from the raw MT output. Our results show that the greater the human interference in the raw MT texts, the greater their distance from the original text and, consequently, their distance from the MT output. This is the case when the raw MT is fully post-edited. In this case, the PF version tends to be closer to HT, and further from MT, PL and original versions, suggesting a great similarity in terms of features between HT and PF.
Together, these results show that simplification, explicitation and convergence features are present on the surface of translated text, although this picture is somewhat blurred for the literature domain.
In spite of the differences between text domains, our results allow us to affirmatively answer RQ2 and RQ3. We consider the differences between the manifestation of features between HT and PE texts as an evidence for post-editese. However, the post-editese features were found more prominently in the light PE performed by either professional or students. Likewise, we consider the differences encountered between domains, translators proficiency levels and types of PE as evidence of the effect of these factors on the way post-editese features are manifested on the surface of the texts. Further, our findings corroborate Toral (2019) in confirming that MT translated texts and PEs are more influenced by the original texts. Like Toral (2019), we hypothesise that this behaviour is due to a priming effect between the MT output and the post-editor resulting in texts that are more aligned between these two. Additionally, our study adds a new finding to Toral (2019) and Daems et al. (2017): a high number of human edits in the raw MT by means of PE results, on the one hand, in a wider distance between the other PE versions, MT, and original; and, on the other hand, in an approximation of the HT version.
The limitations of this study lie in the number of translators and the size of the corpus. Statistical significance tests as well as the convergence experiment would benefit from a wider range of translators and a bigger corpus in order to allow for broader generalisations regarding the differences and similarities found. Nonetheless, the study allowed us to pose research questions that merit exploration in future research. Given that we found, in the news domain, that a greater loss in lexical richness and lexical density was present in HT and PF than in MT texts, does this mean that HT and PF convey less of the original meaning than a MT? Can we be sure that the greater the differences between the original and the translation (as the revealed by HT and PF versions), the higher the quality? To achieve MT quality, should systems be less influenced by the original text but rather convey a more simplified message in terms of lexical and syntactical features? Answering these questions has implications not only for the translation studies field but also for MT quality improvement as it will allow researchers to explore the features that constitute a high-quality MT output.