{'fraction_non_alphanumeric': 0.03282194593361469, 'fraction_numerical': 0.01246150336489107, 'mean_word_length': 4.548014554659073, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Progressive Feature Learning for Realistic Cloth-Changing Gait Recognition

Gait recognition is instrumental in crime prevention and social security, for it can be conducted at a long distance without the cooperation of subjects. However, existing datasets and methods cannot deal with the most challenging problem in realistic gait recognition effectively: walking in different clothes (CL). In order to tackle this problem, we propose two benchmarks: CASIA-BN-RCC and OUMVLP-RCC, to simulate the cloth-changing condition in practice. The two benchmarks can force the algorithm to realize cross-view and cross-cloth with two sub-datasets. Furthermore, we propose a new framework that can be applied with off-the-shelf backbones to improve its performance in the Realistic Cloth-Changing problem with Progressive Feature Learning. Specifically, in our framework, we design Progressive Mapping and Progressive Uncertainty to extract the cross-view features and then extract cross-cloth features on the basis. In this way, the features from the cross-view sub-dataset can first dominate the feature space and relieve the uneven distribution caused by the adverse effect from the cross-cloth sub-dataset. The experiments on our benchmarks show that our framework can effectively improve the recognition performance in CL conditions. Our codes and datasets will be released after accepted.


Introduction
Gait recognition is a valuable tool in video surveillance systems for its outstanding performance in conducting identity verification and abnormal behavior analysis (Larsen, Simonsen, and Lynnerup 2008;Bouchrika et al. 2011;Black et al. 2017). It can identify individuals by the walking patterns of pedestrians from a distance with no contact, even if their face is hidden. However, there still exists a lot of challenges in gait recognition (Yu, Tan, and Tan 2006;Liao et al. 2017;Takemura et al. 2018;Connor and Ross 2018). Among them, a key challenge is to deal with the cloth-changing condition in real-world applications.
Previous research for gait recognition is mostly conducted with the assumption that the sequences of all views and clothes for each subject are available, which, however, is nearly impossible in practice. When collecting the gait benchmarks in the wild (Zhu et al. 2021), it is relatively easier to obtain the cross-view sequences for each subject aided by the techniques such as person re-identification (Chen et al. 2017;Zheng, Yang, and Hauptmann 2016). In contrast, clustering the cross-cloth sequences of the same subject relies on the facial features (Deng et al. 2019(Deng et al. , 2020 which are only available in the front view. And in most cases, the identities in the cross-view and cross-cloth subsets have no overlap. We summarize the composition of a realistic clothchanging dataset in Figure 1. In this work, we propose a new task named Realistic Cloth-Changing Gait Recognition (abbreviated as RCC-GR) to simulate the above setting, which is much more meaningful but has not attracted enough attention. To support our research, we first construct two benchmarks to facilitate the research: CASIA-BN-RCC and OUMVLP-RCC, based on two famous datasets: CASIA-B (Yu, Tan, and Tan 2006) and OU-MVLP (Takemura et al. 2018). Each benchmark contains two sub-datasets: the sub-dataset used to extract cross-view features includes subjects that have view variations but only have the normal walking condition (NM), and the other sub-dataset used to extract cross-cloth features contains other different subjects that have the normal walking condition (NM) and walking in different coats condition (CL), but the sequences view are limited in around 0 • and 180 •1 . In this way, we obtain two benchmarks that can force the algorithm to realize cross-view and cross-cloth with two sub-datasets, which can simulate the real-world applications as much as possible. Second, we propose a new framework that utilizes progressive feature learning to realize gait recognition based on our benchmarks. The framework mainly consists of two methods to map features extracted from the backbone into discriminative feature spaces in a cascaded way, called the Progressive Mapping and Progressive Uncertainty respectively.
Specifically, the previous method used a one-stage module to map features in the cross-view and cross-cloth feature space simultaneously. However, in our benchmarks, the cross-view sub-dataset has richer intra-class diversity and has a wider feature span in the high dimensional space, the cross-cloth sub-dataset has a relatively smaller spatial span in the feature space due to its scarcity in intra-class variety, so if the subjects from the two sub-datasets are mapped together, the spatial space will be distorted . Therefore, our Progressive Mapping applies a two-stage module, which contains a Cross-View Mapping module (CVM) and a Cross-Cloth Mapping module (CCM) to fully span the cross-view feature in the spatial space first, and then extract the cross-cloth feature on the basis. In this fashion, we relieve the uneven distribution in the feature space.
Besides, the Progressive Uncertainty maps each sequence as a Cross-View Gaussian distribution and a Cross-Cloth Gaussian distribution instead of a point. Particularly, mapping each sequence as Gaussian distributions allows the network to increase gradient weight from easy learning crossview sequences and prevent the network from overfitting on the hard cross-cloth sequences. Furthermore, by re-sampling new embeddings from the Gaussian distributions, we can gain more knowledge from the sequence that can be used to classify each identity. Therefore, mapping sequences as Gaussian distributions can further make the feature extracted from the cross-view sub-dataset have more effects in model training.
To sum up, our contributions mainly lie in three folds: • We propose a new setting to simulate the cloth-changing gait recognition problem in practice, which is needed to realize cross-view and cross-cloth with two nonoverlapping sub-datasets.
• We propose a new framework that utilizes progressive feature learning to fully expand the gait representations in discriminative space. In this framework, we design a Progressive Mapping and a Progressive Uncertainty to extract features used for cross-view and cross-cloth in a cascaded way.
• Extensive experiments on our benchmarks prove the effectiveness and outstanding performance of our proposed framework for the cloth-changing gait recognition problem in practice.
Our framework can be employed with off-the-shelf backbones, and we believe our work is significant to the research of gait recognition in real-world applications as well as serving as a foundation stone for further advanced methods.
cross-cloth subset due to the horizontal symmetry.
2 Related Work

Gait Recognition
Model-based Gait Recognition: These methods (Liao et al. 2020;Li et al. , 2021 aim to extract human pose parameters that are more invariant to the view, clothing and carrying variations from videos to realize gait recognition. PoseGait (Liao et al. 2020) exploits the 3D coordinates of joints of the human body as features and designs some handcrafted features based on the 3D pose. Then, CNN extracts the spatio-temporal information from the handcrafted features. GaitGraph (Teepe et al. 2021) uses GCN on the skeleton graph and extracts gait information based on the inherent graph structure. These methods give gait recognition physical explanation and are more suitable with high-resolution inputs.
Appearance-based Gait Recognition: These methods aim to extract discriminative and robust representations from silhouettes. They can perform recognition at lower resolutions and are suitable for outdoor applications. According to the input of the network, the appearance-based model can be roughly further divided into three categories: template-based method (Shiraga et al. 2016), video-based method (Lin, Zhang, and Bao 2020) or image set-based method ). Here we show some representative works of each category. 1) Based on templates: GEINet (Shiraga et al. 2016) uses gait energy image (GEI), the most prevalent image-based gait representation, as input and uses a CNN to extract its potential gait patterns. However, the template cannot encode the temporal information completely, leading to sub-optimal performance. 2) Based on videos: Video-based methods usually use 3D convolution to learn sequence features from both the spatial and temporal dimensions. Wolf et al. (Wolf, Babaee, and Rigoll 2016) uses 3D CNN to extract spatio-temporal information from videos with a fixedlength. To further deal with the variant length of sequences, MT3D (Lin, Zhang, and Bao 2020) uses a frame pooling algorithm to make the input sequences have the same length and designs two branches to extract the spatial-temporal information at multiple scales. Also, GaitGL  uses 3D CNN to attain discriminative representations from both global and local features by a Global and Local Feature Extractor. 3) Based on image sets: GaitSet  argues that the order of the gait silhouettes is not vital to classify each person and first proposes to extract frame-level and set-level features from an unordered set without ranking each frame sequentially like in a video. GaitPart (Fan et al. 2020) further enhances the part-level features with a Focal Convolution Layer and a Micro-motion Capture Module. GLN ) extracts frame-level and set-level features from different stages and merges them by a lateral connection. It also uses a Compact Block to compress gait representations.

Data Uncertainty Learning
In order to improve the robustness and interpretability of CNN, data uncertainty learning has been gradually used in face recognition (Shi and Jain 2019) and person reidentification (Yu et al. 2019) to help the network reject noisy input and avoid false recognition. Data uncertainty learning in face recognition: PFE (Shi and Jain 2019) first proposes to map each face image as a Gaussian distribution, regarding the sequence feature as the mean, and adding another branch to learn the confidence for the sequence feature. DUL (Chang et al. 2020) expands PFE to learn the feature and uncertainty simultaneously. Therefore, the learned uncertainty can affect feature learning by adaptively reducing the negative influence of noisy training samples. ProbFace (Chen et al. 2021) further re-designs the triplet loss, making it aware of the uncertainty, which can be used in our gait recognition task. The triplet loss can help our network focus more on the data from cross-view subdataset, which is the normal walking condition in our task. Data uncertainty learning in Re-ID: DistributionNet (Yu et al. 2019) uses uncertainty learning to deal with noisy samples, such as the samples with the wrong label or outliers data. SCFU (Shi et al. 2022) exploits spatial-wise and channel-wise uncertainty to solve the occluded person reidentification problem.

Our Approach
In this section, we propose a new task called Realistic Cloth-Changing Gait Recognition, which focuses on the cloth-changing problem in practice. In this section, we will first formally define our problem and then introduce a framework with Progressive Feature Learning to fully extract the cross-cloth information in realistic datasets. Specifically, we will describe the details of the two key methods we used in our framework: Progressive Mapping and Progressive Uncertainty.

Problem Definition
We denote the whole training set as (X v ∪ X c ) to represent the two sub-datasets used for cross-view and cross-cloth separately. In X v , each subject only has NM condition, while in X c , NM/CL sequences around front views can be used. It should be noted that X v and X c do not have any overlap. And in most cases, the number of sequences in X v is much larger than that in X c . In the test phase, sequences in all conditions and views will be used for evaluation. Our aim is to alleviate the imbalance in datasets and fully mine the crosscloth information.

Framework Overview
In order to solve this realistic cloth-changing task, We propose a framework that utilizes Progressive Feature Learning. Our framework can be applied to both appearancebased and model-based methods. To simplify the illustration, we take silhouettes as input which is more suitable for realistic low-resolution conditions. According to the sequence quantity proportion in X v and X c , there is an imbalance in benchmarks, which will make the model biased.
If the cross-view feature and cross-cloth feature are treated equally, the uneven data distribution will make them hard to separate from each other and distort feature spaces. So making features from X v domain the feature space first can train the model more robust. In order to achieve this, we Head is a light block used to isolate features of two branches. Progressive Uncertainty constructs two Gaussian distributions with learned mean and variance features to relax constraints on feature mapping.
design Progressive Mapping and Progressive Uncertainty in our framework, and the overview is shown in Figure 2. The training pipeline is as followings. First, the feature extracted from a sequence through the backbone is named f . f will be transferred into two branches, called identity-branch and uncertainty-branch separately. In each branch, Progressive Mapping will use a two-stage module, which contains a Cross-View Mapping module (CVM) and a Cross-Cloth Mapping module (CCM) to first extract the cross-view features from X v , called µ v /σ v , and then extract the crosscloth features µ c /σ c from X c on the basis. Next, Progressive Uncertainty will mapping each sequence as a crossview Gaussian distribution N µ v , σ 2 v I and a cross-cloth Gaussian distribution N µ c , σ 2 c I in feature spaces. Mapping features as Gaussian distributions can make the feature extracted from X v domain the feature space first and then span cross-cloth features from X c . Finally, Uncertaintyaware Triplet Loss (Chen et al. 2021) L tv and L tc are employed to train the whole framework.

Progressive Mapping
Progressive Mapping aims to make features from X v domain the feature space first, making the model learns to separate each identity better. This method has two key components: A two-stage module and Progressive-aware Triplets. Two-stage module: A two-stage module is designed to span the feature used for cross-view and cross-cloth in a cascaded way. In each branch, before the two-stage module, a Horizontal Pyramid Pooling (HPP) ) is employed to first slice the feature horizontally. Then, the twostage module accepts each part of the feature and inputs them to a Cross-View Mapping module (CVM), which is used to first span the feature extracted from X v , and then another Cross-Cloth Mapping module (CCM) is concatenated to span the feature extracted from X c on the basis. Also, a residual connection between the output of CVM and CCM is added to avoid the loss of cross-view information. We take the identity-branch as an example and define the module in the formulation: where µ v is the output from CVM, and µ c is the addition result of the output from CVM and CCM. In this way, the two-stage module can help the model first to learn the identity classification ability better and then improve its crosscloth ability.
Progressive-aware Triplets: Based on the two-stage module, we need to re-design the triplets selection method. We denote a sample triplet as (a, p, n), where a denotes the anchor, p denotes a sample which label is as same as the anchor a, and n is a sample which labels are different from that of a. In our work, we only choose persons from X v in each batch to construct T v = (a, p, n) when calculating the loss behind CVM, making it only extract the cross-view feature.
Then we use the whole batch to construct T c = (a, p, n) when calculating loss behind CCM, so the CCM can make full use of the whole information to cross-cloth as well as maintain the cross-view ability. Also, like in previous research, we select p × k sequences in each iteration, where p represents the number of persons and k is the number of training sequences each person has in the batch. To make the triplets maximum in cross-cloth triplets as well as guarantee the equality, we choose to randomly select (p/2, k) batch from X v , and (p/2, k) batch from X c with k/2 from the NM and k/2 from CL. In this way, we force the network to first learn features from X v and then extract features from both X v and X c .

Progressive Uncertainty
Progressive Uncertainty maps each sequence as a Cross-View Gaussian distribution and a Cross-Cloth Gaussian distribution in the feature spaces instead of a point, which can relax the constraints for the model. The identity-branch accepts features output from the backbone and learns the mean feature, which is used to represent the characteristic of the gait sequence and will be used during inference. The uncertainty-branch is used to generate the variance feature, representing with what uncertainty a feature can represent this sequence, and this branch will be abandoned during inference. In this branch, the original feature f outputs from the backbone is first passed through a Head module, which is a light block with several convolution layers, to make the feature isolated from the identity branch. Also, similar to the identity-branch, we employ another two-stage module to generate two variance features as followings: where σ v is the output from CVM and σ c is the result that added the output from CVM and CCM. To ensure the variances are positive, we apply the Rectified linear unit. With mean feature µ v and µ c and variance feature σ v and σ c , we construct Cross-View Gaussian distribution N µ v , σ 2 v I and Cross-Cloth Gaussian distribution N µ c , σ 2 c I . In this way, the representation of each sequence is not a point embedding but stochastic embeddings sampled from N µ i , σ 2 i I , i = v, c. µ i used during inference can be regarded as sampled from the center point of the Gaussian distributions. The variances measure the uncertainty of the classification result of each stage for the Progressive Mapping, which indirectly benefits the supervision information of the label and optimizes the training process (Shi et al. 2022). The re-parameterization trick: To enrich the information that can be used in classification, we can also re-sample a new embedding from each Gaussian distribution. However, the sampling operation mentioned above is not differentiable and prevents the backpropagation of the gradients flow during training (Chang et al. 2020). In order to solve the problem, we employ a re-parameterization trick (Kingma and Welling 2013) to let the model backpropagate normally like in (Yu et al. 2019;Chang et al. 2020;Shi et al. 2022). A random noise , which is independent of the model parameters, is sampled from a normal distribution and then used to generate the embedding e i , making it to be regarded as re-sampled representations from Gaussian distributions mapped by a sequence.
where e i is the embedding re-sampled with the help of µ i and σ i . Uncertainty-aware Triplet Loss: The mean feature µ i , variance feature σ i , and the sampled embedding e i are used to control the triplet loss. We employ an uncertainty-aware triplet loss (Chen et al. 2021) after CVM and CCM separately: c is the number of triplets. We can also calculate L e tv and L e tv by replacing µ v and µ c with e v and e c . In total, the final loss in our framework can be represent as: L = E(L tv + L tc + L e tv + L e tc ) E(·) means we sum each triplet loss and average them to gain the final loss.

Experiment
In this section, we employ two backbones: GaitSet  and GaitGL  to demonstrate the effectiveness of our framework. Noted that our framework can adopted with model-based or appearance-based backbones to improve their accuracy in RCC-GR task. All models are implemented with Py-Torch (Paszke et al. 2019), and we train with CASIA-BN-RCC on TITAN-XP and trained with OUMVLP-RCC on V100 GPUs.

Datasets Construction
Datasets To support our proposed task, we construct two benchmarks CASIA-BN-RCC and OUMVLP-RCC based on two popular gait datasets: CASIA-B (Yu, Tan, and Tan 2006) and OU-MVLP (Takemura et al. 2018). To construct our benchmarks, we follow 3 principles: 1) The training set collected in real scenes has a sub-dataset that only has NM conditions with view variations, and a sub-dataset has NM/CL conditions only in front views.
2) The two crossview and cross-cloth sub-datasets have no overlap.
3) The sequence number in the cross-view sub-dataset should be larger than that in the cross-cloth sub-dataset. We construct our benchmarks based on these principles and show the statistics in Table 1.
CASIA-BN-RCC The original CASIA-B is a typical gait dataset that consists of 124 subjects. Each subject contains three walking conditions: normal walking (NM-1,2,3,4,5,6), carrying bags (BG-1,2), and walking with different coats (CL-1,2). Each walking condition is uniformly distributed in 11 views, from 0 • to 180 • . Based on previous research Fan et al. 2020), the first 74 subjects are used as the training dataset, and the remaining 50 subjects are used for evaluation. And during the test, NM-1,2,3,4 are grouped into the gallery, and NM-5,6; BG-1,2; CL-1,2 are divided into three subsets according to the waking conditions and are regarded as the probe. However, since the segmentation of CASIA-B is relatively rough, we collected some pedestrian images and trained a new segmentation model to re-segment CASIA-B, and gained CASIA-BN. In order to construct two sub-datasets for CASIA-BN, we first extract the NM, BG 2 sequences with all views in 1-44 identities to construct X v , then extract sequences in 45-74 identities, which walking conditions are NM, BG, CL but views are limited in 0 • /18 • /162 • /180 • to construct X c . The sequence number ratio in the two sub-datasets is 3:1. The test subjects have remained the same as in the previous setting Fan et al. 2020). OUMVLP-RCC OU-MVLP is a large dataset in public which consists of 10307 subjects (5153 subjects for training and the rest 5154 subjects for testing). Each subject contains 14 views, ranging from 0 • to 90 • , 180 • to 270 • . However, each subject only contains one walking condition: normal walking (NM-00,01). During the test, NM-01 is regarded as the gallery, and NM-00 is regarded as probe. To construct X v in OUMVLP-RCC, we extract all sequences of the first 3200 identities in the training dataset. Then for X c , we extract the sequences which views fall in 0 • /15 • /180 • /195 • in the last 1954 identities and abandon some identities without these views, finally get 1944 IDs. To try our best to simulate the condition of walking in different coats, we dilate NM-00 to construct CL-00, dilate NM-01 to construct CL-01 3 . The ratio number between X v and X c also constraints to 3:1. For evaluation, NM-00 is dilated to construct CL-00, and NM-01 is regarded as the gallery, NM-00 and CL-00 are used as probe in different walking conditions with 14 views.

Implementation Details
Input The input size of each silhouette is set to 64×64 for CASIA-BN-RCC and OUMVLP-RCC. During training, we select 30 silhouettes randomly from each sequence as input for GaitSet, and select frames in a fixed order for GaitGL. The batch size is set as (16,16) for CASIA-BN-RCC and set as (32,8) for OUMVLP-RCC. During test, all silhouettes of each sequence are put into model to get their sequence representations. If the probe has the lowest Euclidean distance on all parts with one gallery sequence, it will be assigned with the corresponding gallery label.
Network Structure For the backbone from GaitSet, we only employ the main pipeline, which contains three blocks, and each contains two basic convolution layers. For the backbone from GaitGL, we use the original backbone proposed in . The input and output channels we use for backbones are halved and set as (16,32,64) for GaitSet and GaitGL backbones for CASIA-BN-RCC, for we have less data volume. The channels for each layer we set for OUMVLP-RCC are (64,128,256). For the two backbones, the output dimension of the CVM and CCM module used to generate mean features and variance features is set to 64 for CASIA-BN-RCC and 256 for OUMVLP-RCC. For simplicity, we employ a Fully Connected Layer (FC) for each CVM and CCM module. It should be noted that FC is not the only choice, but we employ it for its simplicity and ease of implementation. And in the head module, we employ two convolution layers (Conv), and each kernel size is 3 × 3, padding is 1, and following the first Conv is the Sigmoid function, following the second Conv is the ReLU. In HPP, we adopt the scale S = 16 to split the feature horizontally. Optimization SGD optimizer is used with a momentum of 0.9 4 . We first train the framework only with the identitybranch, then we load the pre-trained model and train the whole framework with the additional uncertainty-branch to generate the variance feature as well as optimize the mean feature simultaneously. The margin in each triplet loss is set to 0.2 (Hermans, Beyer, and Leibe 2017).
Baselines In order to compare the effectiveness of our framework, we compare our method with several baselines as followings: (1) RCC-Upper. It uses the baseline model and original CASIA-BN and OU-MVLP, which provides an upper-bound performance for the RCC-GR task.
(2) RCC-Base. It gives the results with the baseline model on CASIA-BN-RCC and OUMVLP-RCC with the original framework.
(3) RCC-PFL. It optimizes the model on our benchmarks with our Progressive Feature Learning framework.

Performance Comparison
In this section, we show the performance of our framework on our constructed benchmarks.
CASIA-BN-RCC The performance comparison on CASIA-BN-RCC is shown in Table 2. The probe has been divided into three subsets according to the walking conditions and has been evaluated separately. It should be noted that since our framework aims at improving the precision in realistic cloth-changing condition, we take the 4 Other parameters can be seen in supplementary materials accuracy for CL as the main criteria. From the result, it can be observed that the accuracy of NM and BG is comparable with baselines and upper, but our framework can bring large improvement for the baseline in both backbones for the CL conditions (+6.2% for GaitSet, +3.2% for GaitGL), indicating we can handle the more challenging cloth-changing problem. Table 3, and we also take the accuracy for CL as the main criteria. The result shows that our framework can also have an excellent performance in CL condition on both backbones (+2.9% for GaitSet and +1.9% for GaitGL). Our framework can improve the overall performance not only for CL but also for NM.

Ablation Study
To prove our framework is effective, we conducted several ablation studies with various settings on CASIA-BN-RCC and OUMVLP-RCC systematically. For simplicity, we only use the backbone of GaitSet as our backbone.

Impact of Progressive Mapping
We compare the performance of our Progressive Mapping with another three settings and show the results in Tabel 4. Specifically, all experiments employed with the identity-branch equipped with the two-stage module, the difference lies in the triplet loss. L ni , i = v, c are the normal triplet loss we employed behind CVM/CCM, and normal triplets are selected from both X v Table 3: The rank-1 accuracy (%) on OUMVLP-RCC for different probe views excluding the identical-view cases. For evaluation, the sequences of NM-01 for each subject are taken as the gallery and the probe sequences of the two walking conditions, NM-00 and CL-00, are respectively evaluated.   and X c .
8) The first row shows that only use L nc does not have enough constraints. Compared with our methods, the results in the second row are also slightly poorer, for the negative repel effect of L nv enlarges the intra-class distance of samples in X c , which is larger than the positive aggregate effect of L nc since the diversity is richer in X v , so the adversarial effect will add difficulty to the optimization of CCM. With the three experiments, we can demonstrate that our improvement does not lie in the increased module parameter but in the two-stage module and the design of the triplets.

Impact of Progressive Uncertainty
We show the effect of Progressive Uncertainty in Table 5. The first row shows the result when training with the uncertainty-branch and using the Uncertainty-ware Triplet Loss L tv , L tc . The second row shows the result that we re-sampled one embedding through N µ i , σ 2 i I , i = v, c separately. Our results demonstrate that only using the variance to help optimize the gradient can already improve the performance in the CL condition. That means the performance of our Progressive Uncertainty method mostly depends on the Uncertainty-aware Triplet Loss. In the third and fourth rows, we only calculate σ c , L tc , and still map the sequence as a point behind CVM. The difference is that in the third row, we only use CCM to generate the σ c , and in the fourth row, we use both CVM and CCM to generate σ c , but we abandon the residual connection between CVM and CCM. Our results demonstrate that only mapping each sequence as one Gaussian distribution cannot have the best performance. We will further explain why Progressive Mapping and Progressive Uncertainty can work well in the supplementary materials.

Conclusion
In this work, we tackle the cloth-changing gait recognition problem in realistic application for the first time. We propose a new framework that can be applied with off-the-shelf gait recognition backbones to boost performance in the RCC-GR task. The key methods of our Progressive Feature Learning framework are that we design Progressive Mapping and Progressive Uncertantity to span the cross-view and crosscloth feature in a cascaded way. Extensive experiments were conducted to validate the effectiveness of our framework. It shows that our framework could improve the performance of state-of-the-art methods in walking with different coats (CL) condition in RCC-GR.
The first explanation is that our two methods can optimize the learning direction of our network. Based on the data ratio in our task, the benchmarks are long-tailed. So, in our methods, our Progressive-aware Triplets can make the twostage module first learn the features from X v (head class), then learn to optimize the feature from X c (tail class) as well as maintain the ability to classify head class features. And in this way, the feature can mainly be extracted from the head classes, learning a more robust model for the identity classification task.
Furthermore, Progressive Uncertainty can improve the weight of the head class features in the gradient during training. The head classes have richer intra-class diversity, so they are more easily optimized and can contribute more to the gradient at the beginning of the training process, while the tail class lacks intra-class diversity, so they are hard samples and are not easy to learn and optimize. This property will make the model generate smaller σ for classes and parts which can be easily learned and will generate larger σ for classes and parts which are harder to learn. The σ we generated for each sequence is put in the denominator position of the triplet loss, so the triplet loss is uncertainty aware. The feature with smaller σ will have a larger gradient during training, making the network focus more on these features. While if these head class features are well optimized, the   network will not be misled by the tail class and will learn a more robust model. Figure 2 shows the average σ in the test dataset for the three conditions in CASIA-BN-RCC separately.
We can see that the variance of the easier optimization class NM is relatively smaller than the BG, CL condition. Also, the σ c after CCM is smaller than the σ v after CVM. We also visualize the average value of σ c for the different 16 parts in the feature in Figure 3.
The two pictures indicate that the σ can represent the uncertainty of the classification result of each stage. For classes with more varieties, such as CL condition, they have larger σ, which means they are hard to learn and have large uncertainty and low confidence. For parts in silhouettes, such as the legs parts (11-15 parts in the feature), their dynamic changes are more obvious, so they have larger uncertainty and larger σ. Therefore, our variance can model the un- certainty in both distributions (class type) level and feature (part) level. Also, in Figure 1, we can see that during training, the σ c decreases gradually. In conclusion, the smaller the σ, the better the sequence feature has been learned.
The second explanation is that by applying SVD decomposition for the CCM and reconstructing the CCM only use the first eigenvalue and its corresponding eigenvector, we find the essence of the cross-cloth problem is that the feature space has been compressed towards the direction of the centerline of the NM to CL feature projection.
When the dataset is rich, we can learn a more accurate direction to compress the feature space by CCM, but when the dataset lack information, especially when the CL condition only has views limited in several perspectives, the center point of the CL feature of each identity will not be accurate, as shown in Figure 4.
Since our algorithm can learn a more robust compress di-1 NM CL 0°180°9 0°90°0°1 80°2 Feature Space Figure 4: The feature distribution of NM and CL in feature space. When the sequences in CL condition are rich, the composition direction is along line 1. However, when CL only has sequences in front views, the center point of CL features changes, and the composition direction moves to line 2, which is inaccurate.
rection towards NM to CL, the performance on CL condition can be improved.

Limitations and future work
In this work, we make one of the first attempts toward the Realistic Cloth-Changing Gait Recognition task, which is more practical and meets the industry requirements. Although we have explored the effectiveness of our framework with Progressive Feature Learning, there are still some limitations in our work that need further exploration. First, our work focuses on the realistic cloth-changing problem. To support our research, we construct two benchmarks from existing datasets to simulate the real scene. In fact, with enough funds and time, better datasets collected from the real world can be constructed to further help our task. Second, our work aims to solve the model over-fitting and bias caused by the long-tail property of datasets. However, based on our framework, there is still a large development space for the CL condition to improve. So further research can focus on developing better frameworks to improve the accuracy of CL condition continuously.