{'fraction_non_alphanumeric': 0.028396118721461187, 'fraction_numerical': 0.01091609589041096, 'mean_word_length': 4.988677632984405, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Practical Attacks on Machine Translation using Paraphrase

Studies show machine translation systems are vulnerable to adversarial attacks, where a small change to the input produces an undesirable change in system behavior. This work considers whether this vulnerability exists for attacks crafted with limited information about the target: without access to ground truth references or the particular MT system under attack. It also applies a higher threshold of success, taking into account both source language meaning preservation and target language meaning degradation. We propose an attack that generates edits to an input using a finite state transducer over lexical and phrasal paraphrases and selects one perturbation for meaning preservation and expected degradation of a target system. Attacks against eight state-of-the-art translation systems covering English-German, English-Czech and English-Chinese are evaluated under black-box and transfer scenarios, including cross-language and cross-system transfer. Results suggest that successful single-system attacks seldom transfer across models, especially when crafted without ground truth, but ensembles show promise for generalizing attacks.


Introduction
Recent studies show that natural language processing (NLP) applications are vulnerable to adversarial perturbations, where a small change to the input produces an undesirable change in system behavior, such as a lower-quality translation in a machine translation (MT) system (Ebrahimi et al., 2018;Cheng et al., 2019;Wallace et al., 2019;Cheng et al., 2020;Zhao et al., 2018;Zhang et al., 2021). These adversarial inputs offer insight into model robustness. They also can constitute vulnerabilities that expose everyday technology to malicious actors who would seek to deny and deceive artificial intelligence systems.
Practical concerns must be addressed to determine if these vulnerabilities persist outside of simplified scenarios. Most previous work uses the same ground truth to craft and evaluate an attack and relies on access to the model being attacked, such as model gradients (white-box) or the output of the model (black-box). We ask whether this vulnerability extends to attacks crafted with limited information about the target: without access to ground truth references, model weights or even the outputs of the particular MT system they are attacking. We examine transfer of adversarial examples among eight different MT systems with three target languages. For evaluation, we use a high threshold of success that takes into account both effect on translation quality and loss of meaning in the original text.
We introduce a novel text editing system (perturber) that rapidly generates hundreds or thousands of candidate edits using a compendium of vetted paraphrases scored to match human quality judgments. Adversarial edits are selected according to a configurable optimization trading off preservation of source-side meaning and degradation of target output. To simulate a scenario where a human reference is not available, the selector estimates degradation in translation quality using the change in translation output from a proxy MT system. These attacks meet the threshold for success when the MT system used for selection is matched to the victim model or when an ensemble of MT systems is used to do the targeting. However, we find that examples selected using a single translation model as proxy and ensembles crafted without sensitivity to source-side meaning changes do not often transfer to another victim model above the success threshold.

Practical Considerations
Overwhelmingly, previous work assumes high-information scenarios, using the same ground truth and model to craft and evaluate the attacks, and evaluates adversarial effect separately from the effect on the semantics of the input (Ebrahimi et al., 2018;Cheng et al., 2019;Wallace et al., 2019;Cheng et al., 2020;Zhao et al., 2018;Zhang et al., 2021). We address four considerations in evaluation of machine translation attacks with the purpose of understanding whether these attacks can be crafted in lower-information scenarios and whether the effect on system performance outweighs the degradation of the input text. First, we define our success criterion in a metric-independent way, drawing from Michel et al. (2019), to combine adversarial effect and degradation of the source in a single metric. Second, we calibrate similarity metrics so that one unit of meaning preservation in the source language side is as close as possible to one unit of translation quality in the target language side. Third, we consider whether attacks require access to ground truth to successfully degrade performance. Finally, we address whether attacks crafted using one system can be deployed against another to which it does not have access.

Successful MT Attacks
Effective adversaries do not simply change a system's behavior; they reliably degrade its performance. To attack MT, perturbations aim to maximally decrease translation quality with respect to the ground truth reference. The translations of a set of perturbed source segments should score lower than the originals under some MT metric, such as BLEU or CHRF. However, to ensure that the perturbations haven't trivially reduced translation quality by changing the meaning on the source side, we must also account for the effect of the perturbations on the meaning of the source.
We directly adopt several terms and metrics from Michel et al. (2019). We follow the convention that x and y refer to source and target language sentences, y M is the translation produced by model M , andx andŷ M are the perturbed version of the source sentence and its translation. We measure the translation quality of an attack by the target similarity, s tgt (y,ŷ M ), where y is a gold source reference translation andŷ M is the MT system output on the perturbed input. The effect of a perturbation on the input text is measured by the source similarity, s src (x,x).
An attack degrades the target similarity by d tgt in Equation 1. This is also referred to as target relative score decrease. It is similar to the version found in Michel et al. (2019) except we allow negative values of d tgt if an attack inadvertently makes the translation better. We do this because we do not presume oracular access to a reference translation y at targeting time to decide when to avoid using a particularx for attack. A higher value of d tgt means the output of the MT was more degraded. A value of zero means no degradation. Similarly, an attack degrades the source similarity by d src in Equation 2. Using relative rather than absolute score decreases makes it possible to compare attack effectiveness across models with different original performance.
A successful attack reduces the target side translation similarity more than it reduces the similarity of the perturbedx to x. This is reflected in Equation 3, also following Michel et al. (2019) aside from the difference in d tgt . When success, S, is greater than one, the attack achieved that goal. S values below 1 indicate the source side texts were degraded more than the effect on the translation.
We estimate both source-and target-side similarity using CHRF (Popović, 2015). This metric has been found to be well-correlated to human perception of meaning preservation for varied machine edits (Michel et al., 2019;Merkhofer et al., 2021), and it best matches human perception of machine translation quality at a segment level for the language pairs studied (Mathur et al., 2020).

Calibrating Meaning Preservation Metrics in Multiple Languages
Most semantic similarity metrics are designed and tested to match human judgments in one language, but they generally aren't calibrated to line up with each other across languages. The success criterion in Equation 3 directly compares similarity in source and target languages, but a similarity reduction in the source language needs to be commensurate with the similarity reduction in the target language. Otherwise, perturbations may game the difference between the two scales rather than truly exploiting an MT weakness. A set of examples motivating the differences in CHRF values in different languages can be see in Table 1. This can be replicated for other languages and other metrics, as MT metrics are typically not calibrated across languages, especially not at sentence-level granularity.
Calibration of metrics in different languages relies on common sources of strings with the same distribution of meanings in the two languages. We collect a distribution of s(x i , x j ) values from random strings following the same sampling pattern in each of the languages. We convert those empirical distributions into complementary cumulative distribution functions (CCDF) to work well with log scale. Figure 1 shows the empirical distribution of CHRF scores accumulated using random strings sourced form WMT20 parallel data (Barrault et al., 2020). The samples were synchronized across the languages so the same underlying distributions were reflected in each curve. Using sampled i, j ∈ 1 . . . N from the N sentences available, strings used are i ⌊xlen(i)⌋:⌊ylen(i)⌋ and j ⌊zlen(j)⌋:⌊wlen(j)⌋ , where x < y, z < w ∈ [0, 1).
Conversion of the calibrated scores from the CCDFs to a common language and range is done by linear interpolation. Each curve is approximated with 1000 points as shown in Figure  1. To calculate an associated English CHRF EN for an input Chinese CHRF value, x, we find the closest two surrounding x-axis pairs of the zh CCDF curve and interpolate between them to get y ′ , the estimated CCDF value for that input x. Then the process is performed again using the en curve. We find the two closest y-values on the en curve and interpolate using y ′ to find an x ′ on the x-axis of the en curve. The resulting converted metrics, all calibrated to English, are shown in Figure 2.  Table 1: Examples of high CHRF variance from the WMT20 dataset. σ is the standard deviation of the set of four CHRF scores. Each pair in a set would ideally exhibit the same meaning preservation score.

Crafting Attacks without Reference Translations
Black-box adversarial examples are crafted by probing the victim system for translations, with the goal of finding a perturbed input that minimizes similarity between the system translation and the reference. In the literature, this is typically the same ground truth reference used for evaluation, but in a practical attack, acquiring a human translation for each segment would be prohibitively slow and expensive. We craft perturbations using the system translation of the original source segment in place of a targeted reference translation to simulate a more realistic, low-information scenario where an adversary doesn't have access to a ground truth. In this case, the probes reveal how system behavior changes but not how translation quality with respect to a human reference translation is affected. This allows us to examine whether simply probing the system can effectively predict perturbations to reduce system performance.

Transfer: Using one MT System as Proxy Target for Another
We examine transfer attacks by measuring the effect of each set of black-box perturbations on the other MT systems. Without direct white-box access to the model gradient or black-box access to repeated probes, transfer attacks rely on the nature of language or implicit similarity between systems. When perturbations succeed against another model, the first system can serve as a proxy to craft attacks on the victim system. Intuitions suggest that transfer attacks are less  likely to be effective than black-box attacks, but we want to measure that effect. We also investigate ensemble perturbers, which select edits based on expected performance against multiple MT systems, as an instance of transfer.

Experiments
We present a novel attack mechanism that uses a finite state transducer-based paraphraser to generate paraphrases and then selects the best candidate as the attack. We test two targeting conditions, reference, where the attack is crafted using ground truth translations from the dataset, and MT, where only the system translation of the original source is used. Each set of perturbations is evaluated against the MT system used to craft it (black box) and each of the other MT systems (transfer). Much of the prior work uses reference translations for attack crafting and presents primarily black-box evaluations, but crafting perturbations using only MT outputs and testing transfer to inaccessible systems is a more realistic, low-information scenario. We compare our adversarial FST to the black-box SEQ2SICK approach from (Cheng et al., 2020) under the same conditions. The evaluation metrics are calibrated CHRF, converted into English-equivalent CHRF EN , and Success using calibrated CHRF.

Data
Our experiments use the WMT 2020 test sets for EN-DE, EN-CS and EN-ZH (Barrault et al., 2020). The source for each target language test set consists of the same 1418 English-language segments from the news domain.

MT Systems
Our experiments probe eight trained machine translation systems acquired from the Transformers model zoo (HuggingFace, 2020). mBART English-to-Many is a transformer with multilingual pretraining that is fine tuned to translate from English into many other languages including DE, CS and ZH (Tang et al., 2020). We use separate bilingual EN-DE, EN-CS and EN-ZH models from OPUS-MT (Tiedemann and Thottingal, 2020

Attack: Finite State Transducer-based Paraphraser With Rescoring
We produce adversarial examples by first generating a large portfolio of paraphrasesX = {x 1...n } for the input x, then selecting the best candidate under a configurable mix of source similarity and attack effectiveness. For these experiments, we weight these two factors equally.
To preserve the semantics of an input, our method begins with high-quality paraphrases. We compile 2.3 million equivalence paraphrases from the Penn Paraphrase Database (PPDB) (Ganitkevitch et al., 2013) into a finite state transducer (FST) rewriting input strings. We use a log transform of the PPDB2 score, the estimate of human acceptance included with each PPDB entry, as the weight for the transduction and follow the methods of Stahlberg et al. (2019) to generate a lattice of alternatives for input strings. We minimize the lattice FST resulting from the composition of the input string with the transducer, remove epsilons and determinize, then use the shortest path search. We keep the n-best list of alternatives to use as our candidate edit pool, with n = 1000. It takes an average of 0.085 seconds on one CPU to obtain 1000 alternatives for the input sentences studied in this paper using the pynini toolkit (Gorman, 2016) built on top of OpenFST (Allauzen et al., 2007).
We select one perturbationx per segment per system that balances attack effectiveness and meaning preservation. We estimate both in terms of similarities as measured by CHRF. Meaning preservation is measured by comparing the original source and the candidate paraphrase, s src (x,x). For every translation system M , we obtain translations y M for x, the original source, andŷ M for eachx. Attack effectiveness is estimated by measuring how much the system output differs from the output on the original text, that is s tgt (y M ,ŷ M ), for the MT condition, or by measuring degradation in translation quality s tgt (y,ŷ M ) for the reference condition. For MT system M , we select the candidatex that maximizes f (x, M ) = s src (x,x) − s tgt (y M ,ŷ M ), equally weighting source attack effectiveness and meaning preservation.
Meaning preservation s src (x,x) and attack scores s tgt (y M ,ŷ M ) are scaled prior to selection with a simple Gaussian transform to get them into a comparable range. Without the score transform, the source language similarity scores would tend to be very high compared to the MT similarity scores. This rescaling makes the aggregate optimization more well-balanced.

Ensemble attacks
Ensemble attacks were crafted by evaluating attack candidates using multiple MT systems and averaging the resulting target similarity values, s tgt (y M ,ŷ M ), when performing the attack selection. Mean refers to attacks using all eight systems, f mean (x) = i f (x, M i ). Leave-one-out (denoted loo) refers to averaging all but the victim system's similarity estimate, f loo (x, M j ) = i̸ =j f (x, M i ) where the victim system is M j . The leave-one-out condition simulates attacking an otherwise unknown, inaccessible MT system. Both ensemble techniques realized gains in transfer success count as more systems were included in the ensemble.

Baseline Attack: SEQ2SICK
We use the black-box implementation of SEQ2SICK (Cheng et al., 2020) in the TextAttack python library (Morris et al., 2020b) as a baseline attack on machine translation. This targeted attack generates candidate edits by swapping words for other words that are close in word embedding space. It obtains translations for each candidate from the model and greedily applies one-word changes that minimize the number of words that are present in both the reference and the translation. For the MT condition, we treat the translation of the original source as the reference.
Using a GPU, one attack takes an average of 32 seconds and 285 probes when targeting the reference or 35 seconds and 313 probes when targeting the system translation of the original source.

Results
Success Every set of perturbations degrades the translation quality of every model. All sets of black-box perturbations using the FST-based perturber meet the criterion of success under both targeting conditions. However, many transferred FST-based attacks and many SEQ2SICK attacks under both conditions do not achieve success. Table 2 counts the number of successes over both perturbers under different conditions. Table 4 presents more details for attacks using the FST-based perturber, which is more often successful. Each system's performance on the unperturbed WMT20 dataset, as measured for this study, is reported as original CHRF. Table 2 shows the effect calibration has on success rates. Tuning and measuring performance with calibrated metrics reveals that uncalibrated metrics overestimate success. The systems in the uncalibrated conditions exploited mismatches in the CHRF scales for different languages rather than vulnerabilities of the MT systems.

Effects of Calibration
Referenceless attacks Attacks crafted against the reference achieve a higher margin of success under black-box scenarios and are much more likely to transfer than attacks crafted against the original system output. This suggests that the changes made under the MT condition are more tailored to the errors in the system with which they were crafted, perhaps by further changing parts of the system translation that already do not match the ground truth. Since transfer tends to reduce adversarial effect, the effect of these weaker attacks less frequently outweighs the degradation of the source.
Attack transfer These results don't suggest trends in transfer that correspond to system/language similarity or relative performance. While Allen and Opus-DE were relatively vulnerable to reference-targeted attacks from other systems, this vulnerability doesn't extend to MT-targeted attacks. Adversaries crafted using the multilingual model, mBART, do not transfer better between its different target languages, even though they share model weights.
Ensemble attacks often transfer: The mean ensembles are successful against nearly all individual models and the leave-one-out ensemble attacks successfully transfer in eleven of the sixteen settings. Continuing to add more MT systems to a leave-one-out targeting system would likely increase its effectiveness. Favoring attacks that succeed against more targeting systems leads to better transfer to previously unseen systems.
Example 1: Changing translated day of week (mBART MT, ssrc = 0.94, dtgt = 0.32) original Sacramento police also announced Thursday their internal investigation did not find any policy or training violations. attack Sacramento police also announced Today (thursday) their internal investigation did not find any policy or training violations. Example 3: Unrelated translation (Allen MT, ssrc = 0.99, dtgt = 0.89) original Many readers, including some who work in national security and intelligence, have criticized The Times's decision to publish the details, saying it potentially put the person's life in danger and may have a chilling effect on would-be whistle-blowers. attack Many readers, including some who work in national security and intelligence, have criticized The Times's decision to publish the details, 's saying it potentially put the person's life in danger and may have a chilling effect on would-be whistle-blowers. reference Vieler Leser, darunter auch einige, die für die nationalen Sicherheits-und Nachrichtendienste arbeiten, haben die Entscheidung von The Times, Details zu veröffentlichen, kritisiert und geäußert, dass dadurch wahrscheinlich das Leben der Person in Gefahr gebracht wurde und es einen abschreckenden Effekt auf potenzielle Whistleblower haben könnte. original output Viele Leser, darunter einige, die in national Sicherheit und Intelligenz arbeiten, haben die Entscheidung der Times kritisiert, die Details zu veröffentlichen, sagte, dass sie potenziell das Leben der Person in danger und könnte eine abschreckende Wirkung auf würde -@ be whistle -@ be whistle -@ blowers. attack output Die Times ist eine US-amerikanische Schauspielerin. [The Times is an American actress.]   Table 4: Full FST+Rerank targeted attack results using calibrated metrics. Successful attacks in bold. mean and loo represent targeting with the mean and leave-one-out ensembles. s src measures meaning preservation on the source side using CHRF. s tgt is the MT score of the output sentence, measured with CHRF calibrated to English. L src is the mean edit distance between attacks and originals. Note reference-informed attacks exhibit more conservative edits.
Examples The examples in Figure 3 illustrate some of the range of translation errors given perturbed inputs. They are drawn from black-box FST-based perturbations against the four EN-DE translation systems.

Related Work
The performance of machine translation systems is vulnerable to adversarial examples of several types. Naturalistic and untargeted changes degrade system performance, while remaining largely intelligible to humans (Belinkov and Bisk, 2018). Using word-or character-level permutations, untargeted attacks simply degrade translation quality, while targeted attacks introduce particular errors such as removing or inserting selected words. White-box attacks perturb an input with access to the model's gradients (Ebrahimi et al., 2018;Cheng et al., 2019;Wallace et al., 2019;Cheng et al., 2020) while a black-box paradigm only probes the model's output, typically for salience of portions of the input and scoring of substitutions proposed via heuristics (Zhao et al., 2018;Zhang et al., 2021). Other work crafts attacks based on generally exploitable features of language that are discoverable in training data, such as polysemous words, without probing expected attack success (Emelin et al., 2020). Adversarial examples are crafted with respect to particular models and challenge datasets and they achieve limited success when applied (transferred) to others. A range of text classification adversaries have been shown to reduce the accuracy of models that have different architectures or were trained on different datasets (Song et al., 2021;Ren et al., 2019;Song et al., 2020;Emmery et al., 2021). While transfer effectiveness varies by attack method, it does not reach the level of the matched condition. Several authors show that their adversarial examples, created using white-box attacks on known systems, transfer to some extent to publicly available APIs hosted by Google, Baidu and Bing (Zhao et al., 2018;Zhang et al., 2021;Gil et al., 2019). Emelin et al. (2020) find that their attacks based on dataset co-occurrence reduce the accuracy of several models, but there's little overlap in which examples succeed, with slightly more similarity in sets of examples that are successful on models with the same architecture. White-box, gradient-based attacks can be crafted on models "stolen" via knowledge distillation, despite mismatches in data domain and model architecture (Wallace et al., 2020). Adversarial perturbations typically must conform to perceptual features of an original text. Most NLP attack methods apply one-off perceptual constraints or preferences (e.g. lower number of swaps or similarity among vector representations) but the tradeoff between attack effectiveness and human perception is often unaccounted for, making it difficult to discern when an adversarial effect is the result of perturbations that are easily detected by a human observer (Morris et al., 2020a). Michel et al. (2019) propose a metric for success that balances adversarial effect with the level of meaning preservation of the original.
Paraphrases have recently been used for improving evaluation of MT (Bawden et al., 2020;Thompson and Post, 2020a), for improving MT training (Khayrallah et al., 2020) and multitask MT models have been run in a clever way to generate paraphrases (Thompson and Post, 2020b). The adversarial inputs of Iyyer et al. (2018) are generated using a neural end-to-end paraphrase system.

Conclusion
In this paper, we considered the practicality of adversarial examples for NLP by crafting MT attacks without access to the victim system or ground truth and by measuring those attacks in a way that accounts for both attack effectiveness and source meaning preservation. We find that many attacks that reduce translation quality still fall short of a strict threshold of success. We investigated the ability to transfer attacks across systems and across MT target languages. Attacks that do not have access to ground truth rarely transfer between systems. When they are crafted using ground truth, they transfer more often but we did not observe patterns, like language or system similarity, that allow us to predict when transfer will occur. Our FST perturbation process is able to select edits under configurable constraints that preserve source-side meaning while causing large changes in system output. This is due in part to a high-quality paraphrase generation process relying on millions of paraphrases with scores calibrated to human quality judgments. This selection process is sufficient to degrade translation quality with respect to ground truth. The construction of candidates and attack selection processes do not require a GPU. Ensembles performed the highest rate of successful attacks.
One direction for future work could investigate methods for improving system robustness to attacks of this type. The leave-one-out ensemble was the most reliable attack method we found with at least 50% success rate in all conditions, including transferring attacks to systems it had no previous access to. Building on that success, cultivating it to a robust attack mechanism spanning languages and systems could be another valuable contribution in the future.

Ethical Considerations
There is a risk that adversarial techniques will be used by malicious actors to attack real world NLP systems. We believe that sharing this knowledge allows people who deploy models to account for risk and create safer systems; in particular, we examine how effectiveness measures and techniques reported in recent literature might look under more practical, low-information scenarios outside of academic test harnesses.
Our work is part of a thread in AI assurance that uncovers vulnerabilities and feeds research into mitigation methods, such as model robustness and detection of deceptive inputs.