{'fraction_non_alphanumeric': 0.03950342297487635, 'fraction_numerical': 0.015460709489338007, 'mean_word_length': 5.000393081761007, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Towards Discriminative Representation Learning for Unsupervised Person Re-identification

In this work, we address the problem of unsupervised domain adaptation for person re-ID where annotations are available for the source domain but not for target. Previous methods typically follow a two-stage optimization pipeline, where the network is first pre-trained on source and then fine-tuned on target with pseudo labels created by feature clustering. Such methods sustain two main limitations. (1) The label noise may hinder the learning of discriminative features for recognizing target classes. (2) The domain gap may hinder knowledge transferring from source to target. We propose three types of technical schemes to alleviate these issues. First, we propose a cluster-wise contrastive learning algorithm (CCL) by iterative optimization of feature learning and cluster refinery to learn noise-tolerant representations in the unsupervised manner. Second, we adopt a progressive domain adaptation (PDA) strategy to gradually mitigate the domain gap between source and target data. Third, we propose Fourier augmentation (FA) for further maximizing the class separability of re-ID models by imposing extra constraints in the Fourier space. We observe that these proposed schemes are capable of facilitating the learning of discriminative feature representations. Experiments demonstrate that our method consistently achieves notable improvements over the state-of-the-art unsupervised re-ID methods on multiple benchmarks, e.g., surpassing MMT largely by 8.1\%, 9.9\%, 11.4\% and 11.1\% mAP on the Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT tasks, respectively.

. Given labeled source data and unlabeled target data, our goal is to learn feature representations for recognizing target classes. For this unsupervised domain adaptation task, we propose three technical schemes to learn discriminative target features: (a) clusterwise contrastive learning, (b) progressive domain adaptation (c) Fourier augmentation.

Introduction
Person re-identification (re-ID) is an important task in intelligent surveillance, which aims at identifying the person across different camera views. Recent person re-ID methods have achieved impressive performance owing to the advancement of deep convolutional neural networks (CNNs) [52,67,51,31,6,88,27]. However, the success is mainly attributed to supervised learning over massive human-labeled data. The need of time-consuming manual annotations substantially limits the scalability of re-ID models. Besides, directly applying a pre-trained re-ID model to other new domains may cause significant performance drop due to the inherent data distribution shift across different surveillance cameras. Recently, unsupervised domain adaptation (UDA) has thus attracted much attention to adapt the model learned on a labeled source domain to an unlabeled target domain. Prior unsupervised re-ID methods typically rely on iterative training with pseudo labels generated by clustering algorithms on the target domain [79,70,17,35]. These existing methods have shown promising results but still sustain two main limitations. (1) The label noise may mislead an unexpected optimization direction for network training with the unlabeled target data. (2) The knowledge (i.e. the model ability of distinguishing person identities) learned on source can not be sufficiently transferred to target by simply fine-tuning the source model.
To alleviate these problems, we investigate three aspects to facilitating the learning of discriminative feature representations for better recognizing target classes by (1) reducing the label noise on the unlabeled target data, (2) better transferring knowledge learned from source to target, and (3) adding extra training constraints. Accordingly, we propose a unified framework to achieve these goals. First, inspired by the recent contrastive learning algorithm for unsupervised visual representation learning [21,8,66], we propose a cluster-wise contrastive learning algorithm to learn noisetolerant representations on the unlabeled target data (Figure 1 (a)). Specifically, we construct a momentum-based movingaverage (MMA) feature encoder and build a dynamic queue to provide sufficient negative samples across multiple minibatches for training. Unlike the instance-wise supervision [21,8,66], we incorporate cluster-wise supervision generated by clustering, which is amenable for the high-level re-ID task. Our contrastive learning and feature clustering is performed in an alternating way so that the noise of pseudo labels can be largely reduced. Second, most of existing methods apply a two-stage training process in which the network is first pre-trained on source and then fine-tuned on target. Instead of directly fine-tuning, we adopt a collaborative learning mechanism on both domains with a shared feature encoder (Figure 1 (b)). By gradually decreasing the training weights on source and increasing weights on target, we can better transfer the model ability of distinguishing person identities from source to target. Third, we propose to impose extra constraints in the Fourier space for maximizing the class separability of re-ID models (Figure 1 (c)). We view the amplitude spectrum feature as a kind of nonlinear transformations and compute additional loss functions (e.g, cross-entropy loss) to augment training. In summary, with the proposed method, we can learn better discriminative feature representations and further improve the state-of-the-art performance of unsupervised re-ID.
The main contributions of this paper are summarized as follows. (1) We propose a cluster-wise contrastive learning algorithm to learn noise-tolerant feature representations on the unlabeled target data. The label noise can be largely reduced in the iterative optimization procedure of feature clustering and learning. (2) We propose a progressive domain adaptation strategy to gradually transfer the knowledge learned by the labeled source domain into the unlabeled target domain for unsupervised re-ID. (3) We propose to impose Fourier constraints to further maximize the class separability of the model. We observe the frequency spectrum features can be complementary to the spatial features and beneficial for improving the re-ID performance. (4) Empirical evaluations demonstrate that our method consistently outperforms prior state-of-the-art methods on multiple benchmarks by a large margin. Particularly, using the same ResNet-50 backbone, we surpass MMT [17] by 8.1%, 9.9%, 11.4% and 11.1% mAP on the Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT tasks, respectively.

Related Work
Unsupervised Visual Representation Learning. Unsupervised visual representation learning aims to learn rich feature representations from large-scale unlabeled images, which is also related to our work. The key idea to perform unsupervised learning is constructing pretext tasks with free supervision. Typical methods include recovering the input image by auto-encoders [57,43,77], predicting spatial context [13,40], clustering features [4,3], tracking [61] or segmenting objects [42] in videos and discriminating the instance-wise samples [66,21,8]. Similar to [21], we also build a contrastive self-supervised learning framework to learn representations on the unlabeled target domain. However, in contrast to relying on the instance-wise supervision by maximizing agreement between differently augmented views of the same instance, we incorporate cluster-wise supervision generated by iterative clustering into contrastive learning. We observe that the class information is more suitable to learn discriminative representations for the re-ID task. Moreover, rather than collecting all the samples from the queue [21], we filter out those having the same pseudo-class with the anchor to ensure the quality of negative samples. Contrastive learning is also widely used in many supervised learning methods where training samples are off-the-shelf with labels. In this work, we focus on how to collect meaningful pairs and reduce the label noise in the unsupervised case.
Unsupervised Domain Adaptation. Generic unsupervised domain adaptation (UDA) methods address the closedset problem where the target domain shares the same semantic classes with the source domain. Typical UDA methods focus on reducing the domain discrepancy by aligning data distribution between source and target domains [69,47,74,46,48], training adversarial domain-classifiers to encourage features of source and target domains to be indistinguishable [78,55,28], or learning domain-specific properties [2,19,38,78,5]. In this work, we tackle the more challenging open-set problem of UDA for re-ID, where the classes between the source and target domains are not shared. In fact, our method does not rely on any assumption on the classes. The classes between source and target can be exactly the same, totally different or partially shared.
Unsupervised Cross-Domain Person Re-ID. Although supervised person re-ID methods have achieved great performance on the trained data domain [23,52,67,41,51,31,6,88,24,25,80], the accuracy often drops significantly when directly testing on a different domain. Recently, unsupervised cross-domain person re-ID methods [7,12,49,34,60,83] have attracted much attention to address the problem. Typical approaches [79,70,44,17,71] take a pre-trained model on the labeled source domain as the initialized feature encoder, and further optimize it on the unlabeled target domain by metric learning or unsupervised clustering. Instead of directly fine-tuning the source model, we progressively transfer the knowledge from source to target. Some approaches [17,86,73] apply soft labels for training on target, which can reduce the effect of noise to some extent in the optimization process. NRMT [81] introduces a collaborative clustering to fit to noisy instances. Another line of recent work [75] attempts to learn domain-invariant features from style-transferred images. DG-Net++ zou2020joint disentangles feature space from each domain into id-related and id-unrelated components However, the model performance heavily counts on the image generation quality and how to optimize the class separability of learned representations is often neglected. Recent work of [18,82] jointly optimizes both source and target domains to produce reliable pseudo labels. Our work is related to [18] in the aspect of contrastive learning. The main differences are three-fold.
(1) For the contrastive loss, [18] integrates instance-level, cluster-level and class-level supervision on both domains while we only employ cluster-level supervision on the target data. Besides, we do not rely on additional tricks to select clusters (e.g., independence or compactness used in [18]).
(2) [18] also performs joint learning of source and target domains. Differently, we focus on progressive training by gradually decreasing the source weights and increasing the target weights. (3) [18] only relies on conventional spatial features while we propose to add Fourier constraints for improving the class separability of re-ID models.
Learning in Fourier Space. The discrete Fourier transform (DFT) converts a finite sequence of values into components of different frequencies, which is a classical mathematical transform method and has many practical applications such as digital signal processing and image processing. Its fast computation algorithm of fast Fourier transform (FFT) and the variant of discrete cosine transform (DCT) have been widely used in data compression. Recently, Fourier transform has also been explored for compressing CNNs by grouping the frequency coefficients of kernel weights into hash buckets [9], discarding the low-energy frequency coefficients [63] or training with band-limited frequency spectra [14]. Another line of recent work attempts to learn straight from the compressed representations in the Fourier space [54,20,15,68] for efficient training and inference.
To reduce the discrepancy between the source and target distributions, FDA [72] adopts style transfer in Fourier space by swapping the low-frequency spectrum of source with the target, which shows promising results on semantic segmentation. However, image-level perceptual changes can cause significant deterioration of the performance of person re-ID since it relies heavily on the appearance characteristics of persons. Different from prior methods, we apply 1D FFT for converting the output of network from spatial space to Fourier space, and then combine the losses that computed on the FFT features for network optimization.

Methodology
We denote the source domain data as , where x s i and y s i indicate the i-th training samples and its corresponding class label. The target domain is denoted as where class labels are not available. The goal of unsupervised cross-domain person re-ID is to learn a mapping function f θ (·) to identify the class label (i.e., person identity) for each target image, where θ is the parameters to be learned. The general optimization target Figure 3. For a fair comparison, we first pre-train the networks on the labeled source data, and then fine-tune these networks with different contrastive learning methods on target. "B", "M" and "C" represents "Baseline", "MoCo" and "CCL", respectively.
can be formulated as: where L s and L t indicate the optimization targets on the source and target domains, respectively. λ s (e) and λ t (e) are variables that change over time (e means epoch) to control training on the source and target domains, respectively. Most of existing methods adopt a two-stage optimization pipeline to address this task. That is, the model is first pretrained on source using the ground-truth labels and then finetuned on target using pseudo labels generated by clustering. We formulate the two-stage baseline method as: where L ce and L tri represents the cross-entropy classification and triplet loss [26], respectively. y s andŷ t means the ground-truth class labels on source and pseudo labels on target, respectively.
However, the pseudo labels generated by clustering inevitably contain noise (i.e., wrong labels), which may cause wrong optimization directions during network training.

Cluster-wise Contrastive Learning
In order to reduce label noise on the unlabeled target data, we propose a cluster-wise contrastive learning algorithm, which is inspired by the recent success of unsupervised feature learning [21,62]. In detail, we perform feature clustering and contrastive learning in an alternating manner, towards refining the noisy pseudo labels and updating network weights iteratively. In each round of alternating training, we first employ unsupervised feature clustering (e.g., DBSCAN) to generate pseudo labels and design a  cluster-wise contrastive loss to train the network: where x t p and x t n indicate the positive (i.e., same pseudo class) and negative (i.e., different pseudo classes) samples to x t , respectively.θ represents a momentum-based movingaveraging (MMA) feature encoder to maintain consistency between past and current features, which is updated asθ ← mθ + (1 − m)θ during training. Here, m is a momentum coefficient which is set to 0.99 in our method. We only update the parameters of the regular encoder θ by backpropagation and constrain no gradient back-propagating for the MMA encoderθ. For the anchor x t , we select its positive sample within the current batch and collect past negative samples from previous batches. To implement cross-batch sampling, we build a dynamic queue to memorize the past features N past . However, it is unreasonable to simply take all the past features as negative samples because previous batches may contain positive samples of the anchor. Hence, based on pseudo labels generated in the current round, we drop the instances which have the same class with the anchor from the queue to ensure the quality of negative samples. By collecting sufficient negative samples across multiple batches for training, the network can be better optimized compared to a naive contrastive loss with very limited training samples within a single batch.
We summarize the main differences between CCL with MoCo [21] as follows. First, MoCo relies on the instancewise supervision by maximizing agreement between differently augmented views of the same instance, while we extend it to a cluster-wise version by exploiting the pseudo labels generated by clustering to construct pairs for learning.
We observe that such class information is more suitable for the re-ID task. Second, in contrast to keep updating the queue throughout the training process, we refresh the queue when pseudo labels are updated by a new round of feature clustering, owing to the fact of the class label of a specific training sample is not consistent across different clustering procedures. Figure 3 reinforces our intuition that cluster-wise supervision is crucial for the re-ID performance. MoCo almost fails on both benchmarks. By combining with the baseline (Eq. 2), MoCo still yields inferior results. One can reason that instance-level supervision and its optimization target of MoCo is different from the re-ID task. Training with instance-wise pairs may hinder the learning of feature representations to distinguish different classes. We also find that CCL can boost the baseline by a large margin on both benchmarks, validating the non-trivial design and effectiveness of our method.

Progressive Domain Adaptation
Individually training each domain (Eq. 3) is not optimal for knowledge transfer, especially when there is a large discrepancy between source and target domains. Moreover, when the number of labeled source images is limited, it can easily induce an overfitting trap and hamper the knowledge transfer from source to target.
To alleviate this problem, we propose a progressive domain adaptation strategy to gradually optimize L(θ) from source to target. Specifically, we decrease the source training weights and increase the target training weights over time. Unlike the two-stage training baseline (Eq. 3), we can formulate λ s and λ t as: where w(e) defines a decay policy. For example, a multistep policy can be illustrated in Figure 2. The training process is divided into three phases according to Eq. 5. First, we follow common practice in supervised re-ID to pre-train the model on source (L(θ) = L s (θ)) as an initialization for the subsequent optimization. Second, we jointly train the network on both source and target domains. For the labeled source data, the optimization objective remains the same as the pre-training phase. For the unlabeled target data, the optimization objective is combination of CCL, cross-entropy and triplet losses based on pseudo labels. Third, since our goal is to accurately predict the target classes as possible, we continue to train the network with the target data only in the final phase (L(θ) = L t (θ)). Figure 4 shows the clustering and re-ID performance throughout training. According to the clustering performance, the results imply that our method can gradually reduce label noise and yield cleaner clusters compared to the two-stage baseline. According to the re-ID performance, the results imply that our method can learn better features gradually and achieve higher recognition performance than the baseline.

Fourier Augmentation
Inspired by [53], we consider to impose extra optimization constraints in Fourier space. Specifically, we first apply Fast Fourier Transform (FFT) to compute the real and imaginary components of the 1D CNN output features. We then exploit the amplitude spectrum M(x) = ||F(f θ (x))|| to compute the cross-entropy loss for the target data. In order to better understand the proposed Fourier augmentation scheme, we provide analysis in the following aspects. (1) M(·) can be viewed as a kind of nonlinear mappings. Thus, joint training on the spatial and Fourier features implies that loss functions are computed for different nonlinear features of a training image. We empirically find that such amplitude spectrum features performs better than an extra single MLP layer. (2) Figure 5 visualizes the CNN feature distribution with and without our Fourier augmentation. he qualitative results show that by imposing these extra constrains for training, different classes can be better distinguished. (3) In mathematics, Parseval's Theorem (a special case of Plancherel Theorem) states the relation between a signal and its Fourier transform. In our case, the relation becomes: where D means the feature length. According to this nature, the triplet losses based on Euclidean distance will be equivalent for the spatial and Fourier features. Hence, we only add the cross entropy loss in the Fourier space.

Overall Optimization
The overall optimization objective of our method can be defined as: where γ is a loss weight that balances the spatial and Fourier losses. δ controls the effect of cluster-wise contrastive learning. We compute both of the cross-entropy and triplet losses for L s and L t spa and only compute the cross-entropy loss for L t fre .

Datasets
We evaluate the proposed PDA algorithm on three realworld person re-ID datasets: DukeMTMC-reID [45], Market-1501 [84] and MSMT17 [65]. The DukeMTMC-reID dataset contains 1,812 identities with 36,411 images captured by eight cameras, which splits 702 identities with 16,522 images for training and the remaining images for testing. The Market-1501 dataset consists of 32,688 images of 1,501 identities captured by six cameras, where the training set contains 12,936 images of 751 identities, and the test set contains 19,732 images of 750 identities. The MSMT17 dataset is a large-scale person re-ID dataset, which consists of 126,441 bounding boxes of 4,101 identities captured by fifteen cameras, for which 32,621 images of 1,041 identities are used for training. We report performance on four real-world unsupervised domain adaptation tasks: Duke-to-Market, Marketto-Duke, Duke-to-MSMT and Market-to-MSMT, where the ground-truth labels are provided on source only. We also conduct domain adaptation experiments under the syntheticto-real setting, where PersonX [50] is used as the synthetic source domain. We use the standard mean average precision (mAP) and cumulative matching characteristics (CMC) at rank-1 accuracy as evaluation metrics.

Implementation Details
We use the network (e.g., ResNet-50 [22]) pre-trained on ImageNet [11] as the initial feature encoder. We first train the network for e 1 = 20 epochs on source. For progressive training on both source and target domains, we adopt a kstep policy where the loss weights of source and target are decreased and increased by k times, respectively. We train 30 epochs (i.e., e 2 = 50) in this phase. To learn discriminative features as possible to distinguish target classes, we continue to optimize the model with the target data only in another 30 epochs (i.e., e 3 = 80). The output of MMA encoder is used for inference. For hyper-parameters δ and γ, we conduct parameter analysis to obtain the best choices (δ = 0.1, γ = 0.7) on the Market-to-Duke benchmark and fix them on the other benchmarks. For cluster-wise contrastive learning, we set the temperature parameter τ as 0.07 [21] and set the maximum length of the queue as 1024. The training data are cropped to 256 × 128 and augmented by flipping and rotating with a probability of 0.5. The network is optimized by Adam optimizer with β 1 = 0.9, β 2 = 0.999 and weight decay of 5 × 10 −4 . We set a constant learning rate of 3.

Comparisons to the State-of-the-Arts
We compare the proposed algorithm with the state-of-theart methods on multiple real-world benchmarks for unsupervised cross-domain person re-ID in Table 1. Our method consistently outperforms the existing methods by a large margin and achieves the best mAP performance on all the four benchmarks. Specifically, with the same backbone (ResNet-50) and clustering algorithm (DBSCAN), our method surpasses MMT [17] by 8.1%, 9.9%, 11.4% and 11.1% mAP on the Market-to-Duke, Duke-to-Market, Market-to-MSMT and Duke-to-MSMT benchmarks, respectively. Compared to the other recent unsupervised re-ID methods (e.g., by alternatively training and clustering [70,75,81] or generating synthetic training data by GAN [10]), our PDA method can also obtain superior performance in terms of both mAP and rank-1 accuracy. Compared to the [18] which also uses contrastive learning, our method achieves notable gains on the Market-to-MSMT benchmark, e.g., +9.0% of mAP and +12.1% of rank-1 accuracy. We implement two supervised baselines with the standard cross-entropy and triplet losses using the ground-truth labels of target data. Our method can achieve similar or comparable results in the challenging unsupervised learning scenario. We also compare with existing unsupervised re-ID methods under the synthetic-to-real setting in Table 2. Our method consistently surpasses prior state-of-the-art methods on both benchmarks with a large margin, e.g., outperforming [18] by 5.8% of mAP and 6.2% rank-1 accuracy on PersonX-to-Market. We also evaluate our method on other object re-ID benchmarks [37,36,39] in Table 3. Our method achieves superior performance than MMT and SpCL by a large margin, e.g., +4.3% and +4.2% mAP over SpCL on VehicleID-to-VeRi and VehicleX-to-VeRi, repectively.

Ablation Study
Contributions from Algorithmic Components. Table  1 also shows the relative contributions from each algorithmic component. Our cluster-wise contrastive learning algorithm brings significant improvement over the baseline, e.g., 53.7% vs. 59.6% mAP on Market-to-Duke. With the proposed progressive domain adaptation strategy, we obtain another remarkable performance gains, e.g., 59.6% vs. 67.3%. By adding the extra loss for training in Fourier space, we can obtain consistent improvements by around 2% mAP for all the four benchmarks. By combining our method with other training strategies proposed by MMT (e.g., soft losses and mutual learning), we can further obtain improved performance.
Ablation studies on contrastive learning. First, we compare the proposed CCL algorithm with three contrastive learning methods in Table 4. For a fair comparison, we first Table 1. Performance comparisons on multiple benchmarks for unsupervised cross-domain person re-ID. The supervised baseline is obtained by training with cross-entropy and triplet losses using the ground-truth labels of target data. More stronger supervised baseline is obtained by combining the loss in Fourier space. † means our re-implementation of [17] with the DBSCAN clustering algorithm for fair comparisons. The results of "Ours*" are obtained by combining the proposed method with soft cross-entropy loss, soft triplet loss and mutual learning strategy introduced by [  pre-train the networks on the labeled source data, and then fine-tune these networks with different contrastive losses on target. For the supervised baseline [30], we use the GT labels of target data to generate the training pairs. For other unsupervised contrastive learning methods [66,21], we find they almost fail on these benchmarks. This is because instance- level supervision is often used to learn general feature representations and its optimization target is different from the re-ID task. Directly applying such instance-wise pairs for training may hinder the discriminability of model to distinguish different high-level classes (i.e., person identities). Second, we conduct detailed ablation experiments on our cluster-wise contrastive learning algorithm in Table 5. Without cluster-wise pairs (i.e., using instance-wise pairs as in [21]), the performance drops significantly on both benchmarks (e.g, only 37.3% mAP on Market-to-Duke and 46.1% mAP on Duke-to-Market). Without collecting negative samples in the past iterations (i.e., collecting them from the We also test different sizes of the queue to store the past features for training and find |(N ) past | = 1024 performs best in our setting. Effect of progressive weights. We conduct ablation experiments to show the effectiveness of our progressive domain adaptation strategy in Table 6. The results of (i) show that the performance is poor by directly testing the pretrained source model on target without training. This is not surprising since no knowledge is transferred to the unlabeled target domain. Two-stage training (i.e., first pre-training on source and then fine-tuning on target) obtain inferior results compared to our progressive training strategy. We test different combinations of static loss weights for joint training source and target. The best choice (λ s = 0.2, λ t = 0.8) is still worse than our progressive weights. We also investigate different multi-step policies as well as the linear policy 1 and find that 3-step policy performs best in our experiments.
Fourier space vs. Spatial space. The motivation of our Fourier augmentation to exploit extra feature space to facilitate the network training. We compare the proposed FA with other alternative nonlinear mappings in Table 7. The proposed FA outperforms a single MLP layer (FC + ReLU) on both Market-to-Duke and Duke-to-Market benchmarks, which validates the superiority of the proposed method.  Hyper-parameter analysis. To investigate the importance of loss weights δ and γ, we conduct experiments by changing γ from 0 to 1 under a fixed δ. Table 8 shows that δ = 0.1 and γ = 0.7 perform best on both Market-to-Duke and Duke-to-Market benchmarks.

Conclusion
In this work, we propose a unified framework by incorporating three technical schemes to address the challening unsupervised cross-domain re-ID problem. To learn noisetolerant feature representations, we propose a cluster-wise contrastive learning algorithm by iterative optimization of feature learning and clustering. Instead of simply fine-tuning the pre-trained source model, we adopt a progressive training mechanism to gradually transfer the knowledge from source to target. Furthermore, We impose extra training constraints on the Fourier space for further maximizing the class separability of re-ID models. Our method consistently outperforms prior unsupervised re-ID methods on multiple benchmarks by a large margin. We believe that an extension of this work is to address large variations (e.g., large poses, partial occlusion) in the unsupervised setting.