{'fraction_non_alphanumeric': 0.03826872487218941, 'fraction_numerical': 0.01444383779222713, 'mean_word_length': 4.579617834394904, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Targeted Trojan-Horse Attacks on Language-based Image Retrieval

When a retrieval system expands data, its database is at risk of being attacked. In this paper, we introduce the concept of targeted Trojan-horse (TTH) attacks for language-based image retrieval (LBIR), the first keyword-wise targeted attack against the database of the retrieval system. Specifically, given a specific keyword, TTH generates a QR-code patch that can be applied to a set of different images to gain the targeted Trojan-horse images, which closes to the target keyword in the common space of cross-modal matching of retrieval model. With Uploading the generated TTH images to the database, TTH images will rank high in a normal search, even though the images are completely irrelevant to the query. We evaluate the attacks on standard language-based image retrieval benchmarks (i.e. Flickr30k and MSCOCO) and compare the results retrieved with and without the Trojan-horse images.


INTRODUCTION
Language-based image retrieval is the use of a natural language to retrieve the required images that best match the text query from image database. The core business of language-based image retrieval is cross-modal representation. And the majority of the top-performed methods in benchmarks evaluation are deep neural arXiv:2202.03861v2 [cs.MM] 10 Feb 2022 networks (DNNs) [6,9,12,14,16,23]. These methods mainly aggregate the deep features extracted from pre-trained or fine-tuned DNN models, and then measure cross-modal similarity by calculating the Euclidean distance or cosine similarity.
However, recent studies have shown that DNNs based models are susceptible to adversarial examples [1,10,24]. In the inference stage, these examples will be incorrectly classified as any other label (non-targeted attack) or specific label (targeted attack). For image attacks, there are two methods: perturbations or patches. Perturbations are usually invisible to humans, but usually need to modify the entire image. In contrast, the patch is obvious, but only covers a small area, so it is more feasible in real scenes.
Similar to image classification, adversarial attacks also have been proposed in the domain of image retrieval. For image-to-image retrieval, different approaches such as PIRE [20], UAA-GAN [31], and AP-GAN [30] have been proposed to realize non-targeted imagespecific attack. TMAA [25] is the first targeted attack against imageto-image retrieval and AdvHash [13] extends the targeted attack to class-wise scenario, where the attack method is effective to all queries which belong to a specific class. For cross-modal hamming retrieval, Li et al. [18] first explore cross-modal adversarial samples and then Li et al. [17] investigate the black-box settings. However, two existing works of cross-modal retrieval attack still fall into perturbations on the non-targeted scenario and we aim to let the cross-modal retrieval system return results according to our attack targets.
In the field of image retrieval or cross-modal retrieval, most of the attack methods are to modify the query to achieve the purpose of returning incorrect retrieval results [13,17,18,20,25,30,31]. However, most image search engines allow users to upload pictures, or automatically crawl pictures from the website into the database. Therefore, we can achieve the purpose of interfering with the retrieve process or placing advertisements by uploading the adversarial images to the retrieval database without modifying the user query, which is a back-end attack. To the best of our knowledge, we are the first to explore the back-end attack towards the retrieval system.
As shown in Fig. 2, we upload the images with the adversarial patch to the database of a retrieval system to attack the retrieval system in this paper. A image with the adversarial patch is like a Trojan-horse, so we call it Trojan horse attacks. A Trojan horse has come to mean any trick or stratagem that causes a target to invite a foe into a securely protected bastion or place. A malicious computer program that tricks users into willingly running it is also called a Trojan horse or simply a Trojan.
We propose targeted Trojan-horse Attacks on Language-based Image retrieval (TTH), the first keywords-wise targeted attack against language-based image retrieval, where one single patch with targeted keywords information can be effectively applied to a set of image samples to cause the mismatch when queries contain the targeted keywords. Different from the attack method of disturbing the query, TTH breaks the return list by uploading adversarial images to the retrieval database. Moreover, the attack in TTH is targeted, i.e., TTH wishes the adversarial images ranking as high as possible when retrieving queries including targeted keyword. We formulate this as an images-to-keyword problem and design loss function to increase the cos similarity between embeddings of adversarial images and the contextualized deep embedding of the given keyword. We use the average embedding of different sentences containing targeted keyword to represent the contextualized deep embedding. Then we use the gradient descent method to optimize the patch.
In summary, this paper makes the following contributions: • We propose TTH, the first targeted trojan horse attack against language-based image retrieval, where one single patch with targeted keyword information can be applicable to a set of image samples. • To the best of our knowledge, we are the first to investigate the back-end attack towards the language-based image retrieval system. We upload adversarial keyword images (i.e. TTH images) to the image database, which has a wide range of application scenarios and poses a huge threat to commercial image search engines.

RELATED WORK 2.1 Language-based Image Retrieval
In language-based image retrieval, the objective is to identify the correspondences between a set of text query and images in database, belonging to two different modalities. Since each modality is different, the embeddings produced by specific feature extractors (text, image) will not be inherently aligned. Therefore, the most frequent approach in the literature is to construct a common feature space [2,4,9,11,23,28]. For text representation, BERT [5] is a good text encoder. For image representation, ViT [8] gained superior improvements compared to CNN based model [21,26,27]. Recently, CLIP [23], a large pre-trained model with BERT and ViT as text and visual encoders, becomes popular in language-based image retrieval. In this paper, we use CLIP and finetuned CLIP in target dataset as our deep retrieval model.

Adversarial Attacks in Image Retrieval
For image-to-image retrieval, different approaches such as PIRE [20], UAA-GAN [31], and AP-GAN [30] have been proposed to realize non-targeted image-specific attack. TMAA [25] is the first targeted attack against image-to-image retrieval and AdvHash [13] extends the targeted attack to class-wise scenario, where the attack method is effective to all queries which belong to a specific class. DAIR [3] explores a query-efficient decision-based black-box attack against image retrieval. For cross-modal hamming retrieval, Li et al. [18] first explore tag-to-image adversarial samples and then AACH [17] is proposed to investigate the black-box settings. As summarized in Tab. 1, attack on sentence query modality remains unexplored. Whether it is an attack on image retrieval or tag-to-image retrieval, the usual approach is to push the feature embedding of the target query away from its original region in the feature space. In this way, the images related to the query will not appear in the top list returned by the retrieval. However, these attacks present the same shortcomings: they require test-time queries Sentences with word Iterative images-to-word move … … Figure 2: A conceptual illustration of Trojan-horse image generation. Given a specific word and a set of benign images , we construct a set of Trojan-horse images ℎ, by overlaying a word-specific adversarial patch on each image in . The patch is derived by iteratively making a white-box black on an existing cross-modal matching network to enforce the Trojan-horse images to be more close to the given word in the cross-modal feature space.
to be modified to trigger the attack, which may be unrealistic in practice. Our attack aims to let the searcher see the image specified by the attacker without query modified. This setting is of more practical significance, such as advertising recommendations, avoiding violent image filtering, and so on.

PROPOSED METHOD 3.1 Problem Formalization
We formalize a targeted Trojan-horse (TTH) attack on a given language-based image retrieval LBIR system as follows. Suppose the system, driven by a deep cross-modal matching network N , has indexed a set of 0 images 0 . Each image ∈ 0 has been represented by a cross-modal feature vector denoted by ( ). The system answers an ad-hoc query, expressed in the form of a naturallanguage sentence , by first encoding the query into a cross-modal feature ( ) that shares the same feature space as ( ). The relevance of each image w.r.t to the query is computed in terms of certain (dis)similarity between the corresponding features. The top most relevant images are returned as the search result. A TTH attack is to construct a set of ℎ Trojan-horse images ℎ such that once the indexed collection is expanded as 0 ∪ ℎ , the top images will contain items from ℎ . Consequently, users are shown with images the attacker wants them to see, even though the images can be completely irrelevant to the users' information need.

Trojan-horse Image Generation
We start with a set of ℎ benign images . To simulate a common procedure that expands the database of an image search engine by adding advertising images, we instantiate with such types of images randomly collected from the Internet, see Fig. 3. The Trojanhorse image set ℎ is generated by modifying certain amount of pixels of to embed the TTH attack. In order to let ∈ ℎ be ranked higher w.r.t. a given query , the similarity between ( ) and ( ) shall be larger. However, due to the ad-hoc nature of queries in LBIR, is not known a priori. Directly targeting the query is thus difficult. Alternatively, we aim to construct ℎ for a specific word so that the TTH attack remains effective for a given query that contains . A word-specific ℎ is denoted as ℎ, . In order to let ℎ ∈ ℎ, be more close to in the cross-modal feature space, we introduce a loss as follows where indicates the cosine similarity as commonly used for cross-modal matching [7]. We generate ℎ, by minimizing ℓ ( ℎ , ).
TTH Attack via adversarial patches. As putting a QR code on an advertising image is common, we propose a patch-based TTH attack where the adversarial information is embedded into the QR code yet without affecting its usability. Specifically, we use to indicate an adversarial patch. Such a patch is practically obtained in an iterative manner, so we use to denote the patch after the -th iteration, = 1, 2, ..., , where is a pre-specified maximum number of iterations. Accordingly, an TTH image derived from a specific benign image can be formally expressed as where is a pre-specified binary mask that determines where is overlaid with and ⊙ represents pixel-wise multiplication. In this work, the patch is placed at the top-right corner, see Fig. 2. To make the patch less significant in ℎ, , it is downsized, say to one-tenth of the benign image size. Hence, zero padding on is needed in Eq. 2. The initial state of the patch, denoted by , is fixed to be the QR code of the Wikipedia page of Trojan-horse 1 . Minimizing Eq. 1 alone will introduce distortion to the patch that makes the QR code not scannable. To preserve the code's usability, we add an 2 distance based constrain, obtaining a combined loss as Attack effectiveness where is a positive hyper-parameter that strikes a balance between the attack effectiveness and the QR-code usability. Per iteration, given ▽ as the back-propagated gradient w.r.t. Eq. 3, the adversarial patch is updated as = max(0, min(255, −1 + · ▽ )), with as the learning rate. Note the max-min operation is used to ensure the validity of the pixel values.
Concerning the word embedding ( ), a straightforward choice is to derive ( ) directly from the network N . However, the meaning of a word is context-dependent, subject to the sentence that uses the word. In order to obtain a contextualized embedding of a given word, we gather sentences having , denoted as , from a training corpus, and subsequently perform meaning pooling over the sentence embeddings, i.e., As ( ) is fixed, maximizing the cosine similarity between ( ℎ ) and ( ) means performing an iterative images-to-word move in the cross-modal feature space, as illustrated in Fig. 2. The entire procedure is summarized as Algorithm 1.

Cross-Modal Matching Network
As our proposed method is generic, any cross-modal matching network that produces ( ) and ( ) in an end-to-end manner can in principle be used. We instantiate N with CLIP (ViT-B/32) [23], an up-to-date open-source model for image-text matching 2 . CLIP consists of a BERT for text embedding and a Vision Transformer (ViT) for image embedding. Both ( ) and ( ) have the same dimensionality of 512. The model has been pre-trained on webscale image-text corpora by contrastive learning.

EXPERIMENTS
To verify that the trojan horse images we generated can play an attacking role when retrieve texts contain the target keyword, we make experiments on the commonly used public datasets (i.e. Flickr30k [22] and MSCOCO [19]) on models (i.e. CLIP [23], CLIPflickr and CLIP-coco ). We upload the trojan horse images to the test dataset. When the queries contain the targeted keyword, the trojan horse images returned in the retrieval results rank high, which proves that our attack is effective.

Experimental Setting
Datasets Flickr30k [22] contains 31,000 images collected from Flickr, together with 5 reference sentences provided by human annotators. We follow the split in [29], 1,000 images for validation, 1,000 images for testing and the rest for training. MSCOCO [19] contains about 120,000 images. Each image is described with five related sentences. We adopt the widely used Karpathy split [15], 5,000 images for validation,5,000 images for testing and the rest for training.  Implementation Details. Contextualized deep embedding ( ) is obtained by sampling 500 captions from all captions containing targeted keyword in the training dataset. For patch generation, we set the learning rate equal to 0.01 in all our experiments and perform 300 iterations. If there is no convergence, we half the learning rate and increase the number of iterations by a factor of 2 and re-start. We also set the patch percentage of each sample to be 0.1 and = 0.3.
Performance metrics. We report Recall at 10 (R10), i.e., the percentage of test queries that have relevant items included in the top-10 retrieved items. A successful TTH attack shall decrease R10

Experiment 1. White-box Attack
We summarize the performance of White-box Attack from two aspects R10 of truly relevant images and novel images w.r.t. specific queries. The detailed results are shown in Tab. 2, 3, 4 and 5. The order of keyword is in descending order according to R10 increase of novel images before and after attacking. First, after targeted trojan-horse attack, the R10 of advertisement images retrieved by queries including targeted keyword has increased significantly, which proves that pasting adversarial patch successfully hides the original information of advertisement images and embeds corresponding keyword information. Second, after targeted trojan-horse attack, the R10 of the truly relevant images retrieved by targeted queries are decreased. This is the effect of the higher ranking of the trojan-horse images. From Tab. 6, we can find R10 of truly relevant images and novel images w.r.t. specific queries. In a total of four white-box setting, the average R10 of novel images with targeted trojan-horse attack is 75.4, which shows the effect of our proposed TTH.
As shown in Tab. 3, there is a large difference in the attack performance of different keywords. We assume that some keywords, such as front, are composed of sentences with rich meanings and sparse meaning distribution in the model subspace, so it is more difficult   to construct a unified patch to attack. To prove the assumption, we select two keywords (policeman and swimming) with good attack effects and two keywords (front and run) with poor attack effects. As depicted in Fig. 4, we visualize CLIP-flickr subspace embeddings of all images in flickr30k test dataset, texts including targeted keyword in flickr30k test dataset and trojan-horse images. Because the embeddings of "front" and "run" are scattered in the subspace, it is difficult to put the embeddings of trojan-horse images and all texts including "front" or "run" in subspace close to each other.

Experiment 2. Surrogate-dataset Attack
To explore the generalization ability of our proposed TTH method, we explore Surrogate-dataset experiment. We use R10 of trojan horse images to measure the attack performance. From the Surrogate-dataset Attack experimental results in Tab. 6, we can find that using different datasets for training and test, performance will drop slightly. When LBIR setup is CLIP+Flickr30ktest, if we change the Training data from Flickr30k-train to COCOtrain, the R10 of Novel images will drop from 97.2 to 83.1. When LBIR setup is CLIP+COCO-test, if we change the Training data from COCO-train to Flickr30k-train, the R10 of Novel images will drop from 82.3 to 77.8.

Experiment 3. Surrogate-model Attack
To explore the generalization ability of our proposed TTH method, we explore Surrogate-dataset experiment. We use R10 of trojan horse images to measure the attack performance.
From the Surrogate-model Attack experimental results in Tab. 6, we can find two conclusions. First, using different models to generate trojan horse images and test results have certain attack performance. Second, the cross-model attack between CLIP-flickr and CLIP-coco works good, while the cross-model attack with CLIP is less effective (R10 drops a lot). We speculate that CLIP is a largescale pre-training model, which is quite different from the CLIPflickr and CLIP-flickr that are finetuned in the corresponding data set and corresponding task, resulting in a lot of decrease in attack performance.

Experiment 4. Ablation Study
To explore the influence of and the patch size, we make two ablation studies on CLIP+Flickr30ktest LBIR setup. All other experiment settings remain unchanged as Sec. 4.1.
The influence of . Take motorcycle and policeman for example, the is set to 0, 0.1, 0.3, 1 and 10, respectively. Fig. 5 shows R10 of TTH images and whether QRcode is scannable with the increase of , it can be seen that adversarial patch (QR-code) gradually becomes clear with the increase of from 0 to 1, but R10 unaffected. On the other hand, when set ≥ 0.3, QR code is scannable.  The influence of the patch size. Fig. 6 shows the changing of R10 of TTH images as the adversarial patch size increases. It can be seen that as the patch percentage increases, R10 increases accordingly. When the ratio to benign-image size is greater than 0.1, the performance improvement is relatively flat. What's more, a large adversarial patch size would obscure the main part of the advertisement image. To balance the effect of advertisement and attack performance, we set the patch ratio to 0.1.

CONCLUSIONS
In this paper, We propose TTH, the first targeted attack against language-based image retrieval with adversarial patch. TTH is a back-end attack (i.e. upload TTH images to the database of the retrieval system) and works on a set of queries which contain a specific keyword. To that end, users are shown with images the attacker wants them to see. Our extensive experiments verify that TTH is highly effective at attacking state-of-the-art model CLIP and fine-tuned CLIP in Flickr30k and MSCOCO.