{'fraction_non_alphanumeric': 0.037402470172492215, 'fraction_numerical': 0.01315559287638641, 'mean_word_length': 4.667658139996034, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning to Adapt Classifier for Imbalanced Semi-supervised Learning

Pseudo-labeling has proven to be a promising semi-supervised learning (SSL) paradigm. Existing pseudo-labeling methods commonly assume that the class distributions of training data are balanced. However, such an assumption is far from realistic scenarios and existing pseudo-labeling methods suffer from severe performance degeneration in the context of class-imbalance. In this work, we investigate pseudo-labeling under imbalanced semi-supervised setups. The core idea is to automatically assimilate the training bias arising from class-imbalance, using a bias adaptive classifier that equips the original linear classifier with a bias attractor. The bias attractor is designed to be a light-weight residual network for adapting to the training bias. Specifically, the bias attractor is learned through a bi-level learning framework such that the bias adaptive classifier is able to fit imbalanced training data, while the linear classifier can give unbiased label prediction for each class. We conduct extensive experiments under various imbalanced semi-supervised setups, and the results demonstrate that our method can be applicable to different pseudo-labeling models and superior to the prior arts.


Introduction
Deep neural networks have achieved promising performance in various supervised learning tasks [1]. However, this success crucially relies on large amounts of labeled training data, which inevitably poses a challenge to acquire high-quality annotations, especially in the scenarios (such as medical diagnosis) that require considerable expertise from annotators. A widely-used strategy to alleviate this challenge is semi-supervised learning (SSL) [2], which exploits unlabeled data to help models improve performance, and thus reduce the demand for labeled data.
Among existing SSL methods, pseudo-labeling [3], using the model's class prediction as labels to train against, has attracted increasing attention in recent years. Despite the great success, pseudolabeling methods are commonly based on a basic assumption that the distribution of labeled and/or unlabeled data are class-balanced. Such an assumption is too rigid to be satisfied for many practical applications, as realistic phenomena always follows skewed distributions. Recent works [4,5] have found that class-imbalance may significantly degrade the performance of pseudo-labeling methods. Especially on minority classes, the performance is even worse than the vanilla model trained only with predicted class distribution of FixMatch [6] and ours on the balanced test set. Note that the supervised method trained on the whole training data with ground truth labels, providing an upper bound model for reference. Predicted class distribution is averaged by the predicted scores for all samples.
labeled data [5]. The main reason is that pseudo-labeling usually involves pseudo-label prediction for unlabeled data, and an initial model trained on imbalanced data easily mislabels the minority class samples as the majority ones. This implies that the subsequent training with such biased pseudo-labels will amplify the class-imbalance of training data and further deteriorate the model quality.
To address the aforementioned issues, recent literature attempts to introduce pseudo-label re-balancing strategies into existing pseudo-labeling methods. Such a re-balancing strategy requires the class distribution of unlabeled data as prior knowledge [7,8] or needs to estimate the class distribution of the unlabeled data during training [5]. However, most of the the data in imbalanced SSL are unlabeled and the pseudo-labels estimated by SSL algorithms are unreliable, which makes the aforementioned methods sub-optimal in practice, especially when there are great differences between the class distributions of labeled and unlabeled data.
In this paper, we investigate pseudo-labeling based SSL methods in the context of class-imbalance, in which both labeled and unlabeled data could be imbalanced and their class distributions may differ greatly. In such a general scenario, the current state-of-the-art FixMatch [6] easily suffers from performance degradation. To illustrate this, we train FixMatch on a specific training dataset, where the entire training data are class-balanced while the labeled set is class-imbalanced, as shown in Fig. 1(a). Fig. 1(b) shows that the performance of FixMatch is much worse than the upper bound model (trained with the whole data with ground truth labels), and degrades significantly from the head to tail class. This implies that the class-imbalance of labeled data can readily deteriorate the model training through the pseudo-labeling process. We also visualize the predicted class distribution of FixMatch and the upper bound model in Fig. 1(c), which implies that FixMatch has a strong bias towards the head classes.
To address this problem, we propose a learning to adapt classifier (L2AC) framework to automatically alleviate the influence of online training bias in imbalanced SSL scenarios. Specifically, we propose a bias adaptive classifier which consists of a plain linear classifier and a bias attractor. The linear classifier aims to provide an unbiased label prediction and the bias attractor attempts to assimilate the training bias arising from class-imbalance. We model the bias adaptive classifier in a residual transformation form, so that the two components can be effectively trained. We learn the L2AC with a bi-level learning framework. In detail, the lower level optimization problem leverages both labeled and pseudo-labeled data to update the modulated network with bias adaptive classifier; the upper level problem utilizes an online class-balanced set which is re-sampled from the labeled training data, to tune the bias attractor such that the linear classifier can predict unbiased labels. As a result, the bias adaptive classifier can not only fit the biased training data but also make the linear classifier generalize well towards each class (i.e., tend to equal preference to each class). Together with online pseudo-labeling, the proposed L2AC could effectively deal with class imbalanced SSL.
In summary, our contributions are mainly three-fold: (1) We propose to learn a bias adaptive classifier to assimilate online training bias arising from class-imbalance and noisy pseudo-labels. The proposed L2AC framework is model-agnostic and can be applied to various pseudo-labeling based SSL methods; (2) We develop a bi-level learning paradigm to optimize the parameters involved in our method. This allows the online training bias to be decoupled from the linear classifier such that the resulting network can generalize well towards each class. (3) We conduct extensive experiments on various class-imbalanced SSL setups, and the results demonstrate that our method achieves consistent performance gain over different SSL baselines and significantly outperforms the prior arts.

Related Work
Class-imbalanced learning. Class-imbalanced learning attempts to learn models that generalize well to each classes from imbalanced data. Recent studies can be divided into three categories: Re-sampling [9][10][11][12] that samples the data to rearrange the class distribution of training data; Reweighting [13][14][15][16][17][18][19][20] that assigns weights for each class or even each sample to balance the training data; Transfer learning [21][22][23][24][25][26] that transfers knowledge from head classes to tail classes. Besides, most recent works tend to decouple the learning of representation and classifier [27][28][29][30]. However, it is difficult to directly extend these techniques to imbalanced SSL, as the distribution of unlabeled data is unknown and may differ greatly from that of labeled data.
Semi-supervised learning. SSL aims to learn from both labeled and unlabeled data, including two main lines of researches, namely pseudo-labeling and consistency regularization. Pseudo-labeling [3,31,32,6,33] is evolved from entropy minimization [34] and commonly trains the model using labeled data together with unlabeled data whose labels are generated by the model itself. Consistency regularization [35][36][37][38][39] aims to encourage the model to produce the same prediction when a data point is perturbed or augmented. Despite their success, all of the aforementioned methods are based on the assumption that the labeled and unlabeled data follow the same uniform label distribution. When used for class-imbalance, these methods suffer from significant performance degradation due to the training bias arising from class-imbalance and noisy pseudo-labels. [7,8].
Class-imbalanced semi-supervised learning. Recently imbalanced SSL has been drawing extensive attention. Yang and Xu [40] pointed out that both semi-supervised learning and self-supervised learning can benefit class-imbalanced learning. Hyun et al. [4] proposed a suppressed consistency loss to suppress the loss on minority classes. Kim et al. [5] introduced a convex optimization method to refine raw pseudo-labels. Assume that labeled and unlabeled data share the same distribution, Wei et al. [7] proposed a re-sampling method to iteratively refine the model, and He et al. [8] proposed a Bi-sampling strategy to decouple the learning of representation and classifiers. Different from these methods, this paper aims to learn an unbiased linear classifier which is learned from imbalanced dataset and generalizes well towards each class.

Problem setup and baselines
In general, class-imbalanced semi-supervised learning task involves a labeled dataset D l = {(x n , y n )} N n=1 and an unlabeled dataset D u = {x m } M m=1 , where x n ∈ X are training examples and y n ∈ {0, 1} K are corresponding class labels. We denote the number of training examples of class k within D l and D u as N k and M k , respectively, i.e., In a class-imbalanced scenario, the marginal class distribution of the training data is skewed, which indicates that one of the imbalance ratios γ l := max k N k min k N k 1 and γ u := max k M k min k M k 1 always holds. Note that the class distribution of D u , i.e., {M k } K k=1 is usually unknown in practice due to the unlabeled setting. Given the labeled set D l and unlabeled set D u , our goal is to learn a classification model that is able to correctly predict the labels of test data. We denote a deep classification model and a linear classifier f cls φ , where θ and φ are the parameters of f ext θ and f cls φ , respectively. With pseudo-labeling techniques, current state-of-the-art SSL methods [32,6,33] generate online pseudo-labels for the unlabeled data to augment the training dataset. For unlabeled sample x m , we denote its pseudo-label asŷ m , andŷ m can be a 'hard' one-hot label [3,6,33] or a sharpened 'soft' label [31]. The classification model is then trained on both labeled and pseudo-labeled samples. Such a learning scheme is typically formulated as an optimization problem with objective function L = L l + λ u L u , where λ u is a hyper-parameter for balancing the loss of labeled data L l and the loss of pseudo-labeled data L u . To be more specific, L l = 1 |D l | xn∈D l H (Ψ(x n ), y n ) , wherê D l denotes a batch of labeled data sampled from D l , H is the standard softmax cross-entropy loss; the output probability, and τ is a predefined threshold for masking out inaccurate pseudo-labeled data. For simplicity, we reformulated the loss L as where The pseudo-labeling framework has achieved remarkable success in standard SSL scenarios with class-balanced training data. However, under class-imbalanced setting, the model is easily biased during training, such that the generated online pseudo-labels for unlabeled data can be even more biased. This triggers a vicious circle that further aggregates the class-imbalance issue in the pseudo-labeling process and significantly degrades the performance of minority classes. Moreover, due to the confirmation bias issue [36,41], the learned classification model itself is unable to rectify such a training bias.

Learning to adapt classifier
To alleviate the aforementioned issue, and make full use of the labeled and unlabeled data for training, we design to learn an adaptive classifier which is robust to the training bias (class imbalance and confirmation bias), and thus provides accurate predictions for each class. With such an adaptive classifier, the online pseudo-labeling process can be effective on the unlabeled dataset.
The proposed bias adaptive classifier: The adaptive classifier which we term as F , consists of two modules: a plain linear classifier f cls φ and a light-weight bias attractor ∆f w , as shown in Figure  2. The linear classifier f cls φ aims to provide an unbiased label prediction, which is irrespective to the imbalanced class distribution. The bias attractor ∆f w fits the training bias based on the linear classifier f cls φ . Mathematically, F can be formulated by where z = f cls φ • f ext θ (x) ∈ R K and ∆f w is a nonlinear network with multiple layers. The bias adaptive classifier is presented in a residual transformation form, which has two advantages. On one hand, the nonlinear function ∆f w can theoretically approximate almost any continuous function [42], thus is able to fit the complicated data bias generated in the training phase. On the other hand, the residual learning has been proven successful in easing the training of deep networks [43] and capturing data bias in different visual tasks [44,45].
Learning bias adaptive classifier: With the proposed bias adaptive classifier F ω,φ , the modified classification network can be formulated asΨ = F ω,φ • f ext θ . While F ω,φ aims to decouple the unbiased prediction f cls φ (·) and the bias ∆f w (·), there is no prior knowledge on f cls φ to predict unbiased label prediction and on ∆f w to assimilate the bias. To instantiate such a bias adaptive classifier, in this paper, we propose to learn f cls φ and ∆f w in bi-level learning framework. Specifically, the classification networkΨ is learned to minimize the training loss with both imbalanced labeled and pseudo-labeled data, such that its subnetwork Ψ ofΨ with Ψ = f cls φ • f ext θ is able to minimize the loss of a simulated balanced set (dynamically sampled from the labeled training set).
The bi-level optimization problem can be formulated as where the lower level objective function (denote by L) is and the upper level objective function (denote by L bal ) is where B ⊂ D l is a batch of class-balanced labeled data, sampled from the labeled training set D l and can be easily implemented by class-aware sampling [46]. This objective function aims to turn the bias attractor to make the linear classifier have basically equal preference to all classes.
The bi-level optimization problem (3) can be solved by gradient based method as [47,16]. While to decouple the feature learning and the bias attractor learning, we update the lower level parameters by which indicates that the bias attractor parameter ω is only directly associated to the linear classifier. With gradient based bi-level optimization method and Eq. (6), the parameter ω is updated by where η is the learning rate on ω. Note that in Eq. (7) we need to compute a second-order gradient ∇ ω L bal with respect to ω, which can be easily implemented through popular deep learning frameworks such as Pytorch [48] in practice. The detail of our algorithm is presented in Algorithm 1 and also illustrated in Figure 2. With such a bi-level learning method, the bias adaptive classifier is able to fit imbalanced training data, meanwhile the linear classifier is able to generalize well towards each class such that it can predict labels without bias.
We herein give a brief complexity analysis of our L2AC framework. Since our L2AC introduces a bi-level optimization problem, it requires one extra forward passes in Eq. (5) and backward pass in Eq. (7) compared to regular single-level optimization problem. However, in the backward pass, the second-order gradient of ω in Eq. (7) only requires to unroll the gradient graph of the linear classifier f cls φ . As a result, the backward-on-backward automatic differentiation in Eq. (7) demands a lightweight of overhead, i.e., approximately #Params(f ) × training time of one full backward pass. Therefore, our L2AC is more efficient than most existing bi-level learning methods [47,16,18].

Theoretical analysis
We now present detailed analysis on the debiasing mechanism of the proposed L2AC. Specifically, the update of the parameter ω aims to minimize the problem Eq. (7), and we show how does the value of bias attractor ∆f (·) change with the update of ω. Let P i,k denotes the predicted score P(k|x i ) for training data x i ∈ D l ∪ D u andP j,k denotes the predicted score P(k|x j ) of x j ∈ B, where k ∈ {1, · · · , K} denotes the class categories. We have the following proposition.
Proposition 3.1 Let G i,k be defined as: where Ξ(P i,k ) = , Υ(P j,cj ) = − log(P j,cj ) and c i is the class index for i-th sample, then Eq. (7) can be rewritten as The update of the parameter ω, which aims to minimize the upper level loss, has a consequent effect on the value of the bias attractor ∆f i,k . One can see from Eq. (9) that the sign of G i,k indicates whether ∆f i,k is increased or decreased. Specifically, if G i,k > 0, then Eq. (9) performs gradient ascent for ∆f i,k , and if G i,k < 0, Eq. (9) performs gradient descent for ∆f i,k . The value G i,k in Eq. (8) reveals the similarity between the training sample x i and the whole balanced set B, which means that the value ∆f i,k of bias attractor is adjusted according to the interaction between x i and B. If the responses from x i and B are consistent then ∆f i,k is increased, otherwise ∆f i,k is decreased.

Experimental setups
Datasets. We evaluate our approach on four benchmark datasets: CIFAR-10 [49], CIFAR-100 [49], STL-10 [50] and SUN397 [51], which are broadly used in SSL and imbalanced classification tasks. For CIFAR-10/100, we follow standard protocols in [5]. In detail, a labeled subset and an unlabeled subset are sampled from each dataset, keeping the number of images for each class to be the same. We denote the number of head class within labeled and unlabeled data as N 1 and M 1 , respectively. Then the labeled set is tailored to be imbalanced by randomly discarding training images per class according to the pre-difined imbalance ratio γ l . Without loss of generality, we assume that the numbers of labeled data in each class follow a descending order, and we then have N k = N 1 · γ k−1 K−1 l . Similarly, the long-tailed unlabeled set is created by M k = M 1 · γ k−1 K−1 u with γ u denoting its pre-defined imbalance ratio. Unless specifically mentioned, N 1 = 1500 and M 1 = 3000 are initially set in this work following [5]. The test sets remain unchanged and class-balanced. As for STL-10, we just create imbalance labeled sets in the same way as CIFAR, and its unlabeled set is fully used with M = 100k because of its unknown class distribution. SUN397 is an imbalanced real-world scene classification dataset with 397 classes, which we detail in supplementary materials.
Evaluation Metrics. We adopt the same evaluation metrics as [5] to evaluate the performance. Specifically, we report balanced accuracy (bACC) [52,21] and geometric mean scores (GM) [53,54], respectively, which are defined as the arithmetic and geometric mean over class-wise sensitivity. The results are reported in the form of mean and standard deviation over three random runs.
refines raw pseudo-labels via a convex optimization and CReST [7] that re-balances and aligns the class distribution. Please refer to supplementary materials for more details about these methods.
Implementation details. We follow the experimental setups as in [5]. Concretely, we employ Wide ResNet-28-2 [56] as our backbone network and adopt Adam optimizer [57] with learning rate 0.002 to train the model. The batch size is set to 64 and 128 for labeled and unlabeled data, respectively. The total number of training iterations are set as 2.5 × 10 5 . To evaluate the model, we use an exponential moving average (EMA) of its parameters with a decay rate of 0.999 at each update following [37]. We also follow the evaluation protocols in [37] that evaluates the performance at every 500 iterations and reports the average test accuracy of the last 20 evaluations. More details about the implementations are given in supplementary materials.

Results
We evaluate our L2AC under two different settings: one is γ := γ l = γ u , where both labeled and unlabeled data share the same class distribution; The other is γ l = γ u , where the imbalance ratio of labeled and unlabeled data are different, and γ u commonly is unknown. Case of γ l = γ u . The classification results on CIFAR-10 are shown in Tab. 1, where DARP [5], CReST [7] and our L2AC are all based on Fix-Match [6] for fair comparison. One can see that the performance of many SSL methods is inferior to that of RB methods, even if they use more training data. This verifies that the training bias arising from class-imbalance and pseudo-labels will impair the model during training and results in severe performance degradation. Although Fixmatch outperforms all the re-balancing methods, the resulting model remains biased. This can be observed from its confusion matrix in Fig. 3, where the per-class recall (diagonal vector) degrades considerably from head classes to tail classes. Conversely, our L2AC outperforms all competing methods by a large margin under various imbalance ratios. In particular, L2AC achieves 9.2%/15.9% absolute accuracy gain over the baseline FixMatch for the most imbalanced scenario γ = 150. In Fig. 3, we further present the confusion matrices for imbalanced SSL methods, L2AC provides a relatively balanced per-class recall compared with DARP [5] and CReST [7]. These results reveal that L2AC is able to alleviate the training bias arising from class-imbalance and provides an unbiased linear classifier, resulting in a high-quality pseudo-label estimation as shown in Fig. 4(a). The results on SUN-397 dataset further verify the superiority of our L2AC on practical imbalanced SSL tasks. The comparison results on CIFAR-100 are listed in supplementary materials. Case of γ l = γ u . Tab. 2 summarizes the experimental results on CIFAR-10. Note that CReST [7] fails in this case as it assumes that labeled and unlabeled data shares the same class distribution. We can observe that L2AC consistently improves the baseline FixMatch [6] under various imbalance ratios. An interesting observation is that the performance of FixMatch under γ u = 1 is even worse than γ u = 150. The main reason is that the labeled data under γ l = 100 are class-imbalanced such that the pseudo-labeling process has a strong bias to overfit to the head classes (i.e, prone to predicting incorrect labels for the tail classes), which will damage the entire learning process. This illustrates that more unlabeled tail class samples in γ u = 1 (than γ u = 150) leads to more severe performance degradation. On the contrary, our L2AC is able to eliminate the influence of the training bias, which can predict high quality pseudo-labels for unlabeled data to enhance the entire training process. As a result, our L2AC can achieve significant improvement from γ u = 150 (bACC=75.2) to γ u = 1 (bACC=89.5). Besides, L2AC largely outperforms DARP [5], especially in the case where the class distribution mismatch between labeled and unlabeled data are more severe. For example, our L2AC achieves 7.2%/9.2% accuracy gain for γ u = 100 (reversed), in which unlabeled data follow a reversely ordered class distribution against labeled data. In fact, we can observe from Fig. 4(b) that our L2AC obtains more accurate pseudo-labels and alleviates the imbalance bias for unlabeled data, which probably benefits from a more unbiased linear classifier with basically equal preference for each class as shown in Fig. 1(c). We further evaluate L2AC on STL-10 where the class distribution is unknown. As shown in Tab. 3, L2AC surpasses all the comparison methods by a large margin. This implies that our L2AC can successfully work even if the class prior of unlabeled data is unavailable.

Discussion
Compatibility analysis. We apply L2AC to Pseudo-labels [3] and FixMatch [6] for addressing more various settings. The results in Tab. 4 demonstrate that L2AC can consistently improve the baselines for different label sizes and imbalance ratios. This implies our L2AC can efficiently improve pseudo-labeling SSL methods by learning an unbiased linear classifier. Ablation analysis. We conduct an ablation study to explore the contribution of each critical component in L2AC. We experiment with Fix-Match on CIFAR-10 under two typical setups, namely γ l = γ u = 100 and γ l = 100, γ u = 100 (reversed). We first verify whether the bias attractor is helpful. To this end, we apply the proposed bias attractor to FixMatch (FixMatch w/ bias attractor). It can be see from Tab. 5 that bias attractor helps improving the performance by a certain amount, which indicates that the bias attractor is effective, but not significant. As analyzed in section Section 3.2, there is no prior knowledge for the bias attractor to assimilate the training bias in this plain training manner. Next, we study how important the role the bi-level optimization plays in our method. Instead of using bi-level learning framework, we disengage the hierarchy structure of our upper level loss and lower level loss, and reformulate a single level optimization problem as L + λL bal . The model can be viewed as a degenerate version of L2AC, it can be seen from Tab. 5 that this single level optimization provides substantial performance gain over FixMatch w/ bias attractor while still inferior to our L2AC. This can be illustrated by Prop. 3.1, i.e., the bi-level training involves second-order gradient that facilitates the interaction between each training sample and the whole balanced set B, which helps the linear classifier predict unbiased label and bias attractor capture training bias.
Understanding from logit adjustment. The L2AC proposes a dynamic logit adjustment mechanism to some extent, since the logits output by the linear classifier are adjusted by the designed bias attractor. Different from Logit Adjustment (LA) [58], our L2AC sample-wisely adjusts the logits in imbalanced SSL problem with no need for training data distribution, which, however are required to infer the class-wise offset to logits in LA. Since the distribution of the training data (including labeled and unlabeled data) is unknown, it is hard to directly LA to imbalanced SSL. As a compromise, we use the distribution of labeled set to adapt LA loss [58] for training FixMatch. As shown in Tab. 5, FixMatch w/ LA is not comparable to our L2AC, as LA uses class prior of labeled data to adjust the logits, which is not able to assimilate the complicated training bias for imbalanced SSL.
Broader impacts. In this work we develop a bi-level imbalanced SSL framework to help pseudolabeling based models mitigate the impacts of training bias. The method has great potential to improve the fairness of the classifier, and thus benefits ones faced with heavily imbalanced data in realistic applications, e.g., autonomous vehicle and disease diagnosis. On the flip side, the methods may potentially be used nefariously for manipulating the class-wise accuracy by changing the distribution of B in Eq. (5). Therefore, it is critical to make sure that the proposed method is used for the right purpose in practice.

Conclusion
In this work, we propose a bias adaptive classifier to deal with the training bias problem in imbalance semi-supervised learning tasks. The bias adaptive classifier is consist of a linear classifier to predict unbiased label and a bias attractor to assimilate the complicated training bias, and it is learned with a bi-level optimization framework. With such a tailored classifier, the unlabeled data can be fully used by online pseudo-labeling to improve the performance of pseudo-labeling methods. Extensive experiments show that our proposed method achieves consistent improvements over the baselines and current state-of-the-arts. We believe our bias adaptive classifier can also be used for more complex data bias other than class imbalance. The source code will be made publicly available.