{'fraction_non_alphanumeric': 0.03712928829995683, 'fraction_numerical': 0.011623659127893461, 'mean_word_length': 4.648471206152692, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Over-Fit: Noisy-Label Detection based on the Overfitted Model Property

Due to the increasing need to handle the noisy label problem in a massive dataset, learning with noisy labels has received much attention in recent years. As a promising approach, there have been recent studies to select clean training data by finding small-loss instances before a deep neural network overfits the noisy-label data. However, it is challenging to prevent overfitting. In this paper, we propose a novel noisy-label detection algorithm by employing the property of overfitting on individual data points. To this end, we present two novel criteria that statistically measure how much each training sample abnormally affects the model and clean validation data. Using the criteria, our iterative algorithm removes noisy-label samples and retrains the model alternately until no further performance improvement is made. In experiments on multiple benchmark datasets, we demonstrate the validity of our algorithm and show that our algorithm outperforms the state-of-the-art methods when the exact noise rates are not given. Furthermore, we show that our method can not only be expanded to a real-world video dataset but also can be viewed as a regularization method to solve problems caused by overfitting.


Introduction
The current deep learning breakthroughs have largely been due to 'data'; thus, many researchers in both academia and industry endeavor to obtain considerable data. One of the most popular methods to collect data today is crowdsourcing [27,38] (e.g., Amazon Mechanical Turk) because it is cheap and quick. However, these data inevitably contain some proportion of noise in the labels, owing to perceptual ambiguity, human errors, or errors from automatic annotations. These incorrectly labeled data negatively affect the performance of a trained model. Hence, deep learning suffers from noisy labels that are corrupted from ground-truth labels and are incorrectly labeled. Due to the increasing need to handle noisy-label problems in a massive dataset, learning with noisy labels (LNL) has received much attention in recent years [6,9,10,18,25,[34][35][36]40].
The research on LNL can be divided into two approaches. One approach is to directly train a robust model on noisy data, and the other is to find and use only clean data for training. The first approach aims to design noise-tolerant loss functions, architectures, or training schemes [4,23] to estimate the noise transition probability. However, estimating the noise transition probability is challenging, especially when numerous classes exist. Therefore, we focus on the second approach that aims to detect the noisy labels for training on the selected clean instances [6,7,10,12]. Regarding the second approach, most recent studies [6,10,12,37] have focused on finding small-loss instances before a deep neural network (DNN) overfits the training data. These works regard smallloss instances as clean labels based on the observation that DNNs memorize easy instances first and gradually learn more difficult instances [2].
Although these methods achieve a robust classification accuracy, some limitations exist. For example, to prevent overfitting, Co-teaching [6] trains two neural networks simultaneously and the networks teach each other about smallloss instances. However, training two neural networks simultaneously demands vast memory requirements, especially for larger models frequently used these days, such as the deep ResNet [8] and 3D convolutional neural networks(ConvNet) [11]. For a similar purpose, MentorNet [12] has devised an extra network pretrained on clean validation data or a predefined curriculum. However, prior knowledge, such as large-scale clean validation data or a teacher network with a predefined curriculum, is difficult to obtain in practice.
In this paper, instead of aiming to find small-loss instances with the guidance of additional architecture or prior knowledge, our motivation stems from the following question: If a model eventually overfits noisy labels, then can we infer the noisy labels from the trained model itself using its overfitted property? From this motivation, we propose a method to identify overfitting on individual data points. We show how this method can be used to spot incorrect labels, remove them iteratively, and thus avoid overfitting noisy data. To this end, we focus on two crucial properties of an overfitted decision boundary. First, the decision boundary that deviates from the ideal boundary by a noisy-label instance will be sig-nificantly changed when the noisy-label instance is removed. Second, the overfitted model predicts poorly on unseen data; thus, mislabeled data affect the model to classify new data with incorrect labels. From these observations, we present two novel criteria that measure the abnormal influence of a training sample. The overfitting score on the model measures how a training sample affects the change in model parameters, and the overfitting score on the data estimates how inconsistently it affects the classification of a small number of clean validation data. Based on the criteria, we propose an iterative algorithm called Over-Fit that removes the noisy-label samples and retrains the model alternately until no further performance improvement is made.
Through extensive experiments on multiple benchmark datasets, we show that our algorithm successfully detects the noisy-label data, even when the actual error rate is unknown. As a result, our algorithm outperforms recent methods in image classification under various noisy circumstances. In addition, we expand our method on the realistic video dataset, HMDB-51. Furthermore, we show that our method can be regarded as a regularization method. In summary, our main contributions are: • A novel algorithm for detecting noisy-label samples by employing the overfitting property of individual points.
• Experimentally showing that the proposed method can detect noisy-label samples on various datasets including synthetic image data, large-scale real image data, and real-world video data.
• Showing that our method can be used as a regularization method for detecting individual samples affecting overfitted decision boundaries.

Learning with Noisy Labels
Learning with noisy labels has two main research direction. One is to directly train a robust model on noisy labels, and the other is to find and use only clean labels for training.
Noise-tolerant Approach: The noise-tolerant approach aims to train a robust model on a noisy-label dataset without removing the noise. Some methods design noise-robust losses [19,31,34,41], and others attempt to correct losses [20,23]. Despite their theoretical justification, these approaches require mathematical assumptions or prior knowledge, such as noise rates and noise transition matrices, which make them challenging in practice. Although Goldberger et al. [4] attempted to estimate a noise transition matrix by adding a noise adaptation layer, it is difficult to estimate the noise transition probability when numerous classes exist.
Noise-cleaning Approach: Most noise-cleaning approaches focus on finding small-loss instances before overfitting because DNNs learn easy samples first and gradually learn difficult samples [2]. To prevent overfitting of a neural network, some methods simultaneously train two neural networks and select small-loss instances [6,18,37], while others train a network guided by a teacher network [12]. Meanwhile, the O2U-net [10] adjusts the learning rate to take the model from overfitting to underfitting cyclically and records the losses of each sample during the iterations. The main difference between these methods and our approach is that, while they struggle to prevent overfitting and regard small-loss instances as clean instances, we leverage the overfitting property and determine the most abnormal influential instances to rule out.

Influence Function
Finding influential instances in a dataset has been studied for decades in robust statistics [3,5]. Recently, a few attempts have been made to apply the idea to neural networks [1,14]. The influence function is a good measure to check robustness, but it has not been widely used in DNNs because few suggestions on how to use the influence function have been made. First, it is difficult to interpret the results of the influence function because the result is a k-dimensional vector, which is the number of model parameters. Koh and Liang [14] used influence functions to understand the effect of a training sample on a test sample. However, it is difficult to find the pattern on the whole dataset or the model from the individual influence of each training sample on a test sample. Recently, Hara et al. [7] proposed stochastic gradient descent (SGD) influence that can infer the influential instances for models trained with SGD. However, this method is limited to optimization by SGD and must store the parameters of the model at every step, which requires substantial memory for DNNs. In addition, how many influential points to remove is not well defined (i.e., the top-m influential instances are removed). We propose a novel method to use the influence function to identify overfitting and solve LNL problem.

Method
Our idea is to leverage the property of an overfitted model. First, we present the observations that motivated our method in Section 3.1, and review the influence function in Section 3.2. Then, we describe two novel criteria in Sections 3.3 and 3.4. Finally, the automatic data cleaning algorithm is explained in Section 3.5.

Observations Motivating the Work
In this section, we explain how we can locate noisy labels from an overfitted model. It is known that DNNs have a high capacity to memorize data, and DNNs learn "easy" data first and can memorize "hard" instances as training processes [2]. Thus, in noisy-label circumstances, DNNs learn the pattern of clean labels first and eventually fit the noisy labels. Moreover, DNNs can learn feature representations of the data well. Common features over broad classes are learned in the lower layers, and the class-specific features are learned in the upper layers [39]. Then, to fit the mislabeled data against the feature representations learned from the clean data, overfitting would likely occur more frequently in the upper layers and the final fully connected layer, which largely contribute to the decision boundary. Hence, the noisy-label data are critical to the degradation of the classification performance.
In this perspective, our objective is to locate the noisy-label data in the overfitted region around the decision boundary.
To this end, we need to answer a question: How can we find the overfitted region and determine the noisy labels? Figure 1 illustrates a toy example to explain our criteria. The red and blue points belong to different classes for binary classification, and the pink and light blue background indicate the ground-truth feature embedding space. The border line between them is the decision boundary. The decision boundary is distorted by overfitting the noisy-label data, as shown by the black line. In Figure 1(a), the model (black line) overfits the noisy labels (× mark). When the noisy label (× mark) is removed, the trained model is substantially changed (Figure 1(b)), that is, the noisy label can exert great influence on the overfitted model.
In addition, we can use clean validation data to evaluate whether a training sample is in the overfitted region. We consider a few validation data points ( marks) as shown in Figure 1(c). Because the fitted decision boundary is distorted toward the blue region to include the noisy label (× mark), the enclosed by a red dotted circle is wrongly classified into the red class. Thus, the noisy label (× mark) negatively affects the marks enclosed by the red-dotted circle in Figure 1(c)). Meanwhile, the sample × positively influences the validation samples upper the line (blue-dotted circle). The same claims can apply to the validation samples in the red category in Figure 1(d). Therefore, the noisy label is likely to have a significantly inconsistent influence on the clean validation samples, although their distances are near each other.
From this observation, we present two novel criteria that measure the abnormal influences of a training sample. One is to measure how much it affects the change in model parameters, referred to as the overfitting score on the model, and the other measures how inconsistently it affects the classification of clean validation data, which is referred to as the overfitting score on the data. In the following sections, we describe the details of the criteria.

Influence Function
To derive the criteria, we use the concept of the influence function [5], which is a measure of the dependence of the model parameters on a training sample from robust statistics. The influence function estimates the change in the output of the model when a sample point is excluded. Koh and Liang [14] adopted the influence function for DNNs.
Consider a classification problem with n training data (x 1 , y 1 ), · · · , (x n , y n ), where x i is the i-th training point (e.g., an image) and y i is its label. Let f (x, θ) denote a model parameterized by θ and L(y, f (x, θ)) be the loss for a training point (x, y). Given the empirical risk θ)), the optimal parameter that minimizes the risk isθ def == argmin θ R(θ). The influence of a training point (x, y) can be efficiently approximated by the parameter change if the distribution of the training data at the point (x, y) is slightly modified. Then, the influence of (x, y) on the parameters of the trained model has been presented in [14], which is denoted by where is the Hessian and is positive definite by assumption.
Using the influence on the model parameters in (1), Koh and Liang derived the influence of a training sample (x i , y i ) on a test sample (x t , y t ) as We extend the work by Koh and Liang to identify overfitting on individual points for detecting noisy labels.

Overfitting Score on the Model (O M )
I M (x;θ) can be used to estimate the effect of a noisy label on an overfitted model (Figure 1(b)). However, I M (x;θ) is a p-dimensional vector, where p is the number of parameters. Thus, to measure the strength of the influence of a training point (x i , y i ), we use I M (x i ;θ) as a metric. Furthermore, to determine the outliers, we normalize I M (x i ;θ) so that the overfitting score on the model O M (x i ;θ) can produce a relative score with a higher influence than the average as where µ(·) and σ(·) represent the mean and standard deviation, respectively. Therefore, O M (x;θ) is used to determine the noisy-label candidates in our iterative algorithm in Section 3.5.

Overfitting Score on the Data (O D )
In contrast to a well-generalized decision boundary, an overfitted decision boundary by a noisy-label sample makes the noisy-label sample inconsistently affect clean validation samples, even though the validation samples belong to the same class (Figure 1(c) and 1(d)). Here, an influence of a training sample on a validation sample indicates how much a classification result of the validation sample changes after removing the training sample. Therefore, we define the overfitting score on the data as the within-class influence consistency of a training sample x t i on m clean validation samples x v 1 , · · · x v m in each class. For this, we utilize (2) as I D (x t i , x v j ;θ). This function represents the amount of validation error on (x v j , y v j ) after (x t i , y t i ) is removed from the training set. To measure the within-class influence consistency of x t i to the kth class, we propose to use the variance of I D (x t i , x v j ;θ) for all x v j in the validation set of the kth class, which is denoted by . Then, to estimate samples with relatively high influences, we normalize σ k (I D (·;θ)) and obtain the overfitting score on the data O k D (x t i ;θ), as x v ;θ))) .
(4) We will justify the choice of variance operator in the Experiments section.
The overfitting score of x t i on the validation data in the kth class, O k D (x t i ;θ), is used to determine the final noisy training samples to rule out. By detecting the training data whose O k D (x t i ;θ) exceeds the threshold β, we can filter out the highly suspicious training data whose influence inconsistently fluctuates beyond the average.

Over-Fit: Iterative Algorithm for Noisy-Label Detection and Retraining
Our algorithm iteratively detects noisy-label samples based on the O M (x t i ;θ) and O k D (x t i ;θ), and retrains the model until its performance improvement saturates. The iterative design allows successful noisy-label detection from the overfitted model, even under high noise-level circumstances because it removes the most influential samples in priority among many confusing candidates and continuously improves the model iteratively. As the model improves, our algorithm can incrementally find noisy data having abnormal influences that could not be detected in previous iterations.
Given a training dataset X t that includes noisy labels and a small portion of a clean validation dataset X v , let n be the number of training data, K denote the number of classes, and V be the number of validation samples in each class. Then, the number of total validation samples becomes K ×V . First, we train a model on the whole training dataset following the common machine learning procedure until the accuracy in the validation dataset remains stable (overfitted). Then, we calculate O M (x t i ;θ) for the whole training dataset X t . We aim to detect noisy labels that have a high score for both O M (x t i ;θ) and O k D (x t i ;θ). For efficient searching for the noisy labels, in each iteration, we first compute O M (x t i ;θ). Then, we compute O k D (x t i ;θ) for the training samples whose O M (x t i ;θ) are higher than a threshold of α. Among them, the training samples with the kth O k D (x t i ;θ) exceeding the threshold β are assigned to be noisy candidates because they have inconsistent influences to the kth class validation data.
If a noisy candidate is inconsistent for the validation datasets of more than γ classes in common (i.e., more than γ classes have consensus that the noisy candidate is inconsistent), the noisy candidate is decided to be a noisy-probable sample. After removing all the noisy-probable samples, the model is retrained using the new training set. If a meaningful improvement in the accuracy of the validation data occurs, the noisy-probable samples are eliminated from the training set, and the algorithm is repeated. Otherwise, the noisy-probable samples are not removed, and the algorithm stops. Algorithm 1 outlines the full procedure for cleaning noisy data.
After the algorithm finishes, new labels of the removed samples are predicted by the classifier in the last iteration. Simply, we replace the labels of the noisy data with the newly corrected labels. Then, among the corrected training data, the new clean dataset includes only the data whose softmax outputs are higher than S (prediction threshold). Then, the model is newly trained on the new clean dataset and is evaluated for the test dataset.

Experimental Settings
Datasets: We conducted experiments on multiple benchmark datasets including the CIFAR-10, CIFAR-100 [15], Dog vs. Fish images from ImageNet [24], Clothing1M [33] and HMDB-51 [17] (Table 1). CIFAR-10 and CIFAR-100 are the most popular datasets used for evaluating noisy labels in the literature [6,10,12]. We used only 500 randomly sampled data as a clean validation set to compute the overfitting score for CIFAR-10 and CIFAR-100, which is the fairly small number. For visualization analysis on the property of the proposed criteria, we used the two-class dataset, Dog vs. Fish. Also, we evaluated our method on a large real-world dataset Clothing1M, which includes about 38% real noisy labels. Furthermore, to illustrate the applicability of our algorithm to video streams, we experimented on HMDB-51, a popular dataset frequently used in video action recognition [26,32].
All CIFAR-10, CIFAR-100 and Dog vs. Fish include the noisy-label data generated by symmetry flipping in [6], where the labels that are sampled with a probability of ε Hyperparameter Settings: The algorithm (Section 3.5) has three hyperparameters: the thresholds of α for O M , β for O D , and γ for the consensus of {O k D }. Because O M is a normalized overfitting score, the value of an influential sample (noisy label) is greater than the average (i.e., 0). Thus, α was set to zero. To show the impact of β and γ, we vary them with different values. The search space of hyperparameters is β ∈ {0, 0.5} and γ ∈ {0.2K, 0.5K, 0.8K} (see the experimental result in Supplementary Material). According to our observations, in the early stage, removing only highly influential samples is more advantageous. Hence, for cleaning only highly influential samples, we set the threshold β to high as 0.5 and the class-consensus threshold to relatively high as 0.8K for CIFAR-10 and 0.5K for CIFAR-100 (K is the number of total classes). To filter out a large number of noisy samples, we reduced the thresholds for β and γ to low values as 0 and 0.2K, respectively, from the third iteration. Lastly, we set the prediction threshold S to 0.8. Implementation Details: We used PyTorch [22] to implement and train all the models in the paper. For the backbone networks to extract features, we used ResNet-34 [8] for CIFAR-10, CIFAR-100, and Clothing1M whereas Inception-v3 [29] was used for Dog vs. Fish and HMDB-51. For the overfitted classifier, the last fully connected layer in the networks was used to measure the overfitting scores O M and O D . Initially, the networks were trained for 150 epochs with Stochastic Gradient Descent (SGD) (momentum=0.9). For all CIFAR datasets, we set the initial learning rate to 0.01, and dropped it by a factor of 0.1 after 30 and 100 epochs. For each retraining iteration, we resumed training for 30 epochs, where the learning rate at start was set to 0.1, and was dropped by a factor of 0.1 after 1, 20, and 25 epochs. By increasing the learning rate high at the first epoch in each retraining iteration, we could encourage the network to explore and form a new overfitted classifier. We conducted all the experiments on a NVIDIA GTX 1080Ti GPU with Intel i5-2500 CPU. We include more implementation details of all the experiments in Supplementary Material.
Evaluation Metrics: For the evaluation, we used precision, recall, the final noisy-label ratio, and the classification  accuracy. Precision is the ratio of true noisy labels among the samples that the algorithm classifies as noisy. Recall is the ratio of the detected true noisy labels among the total true noisy labels. The classification accuracy is the ratio of correct classifications by a classifier. has the best performance, which means the overfitting scores create synergy when combined. Likewise, using both criteria has the best performance on CIFAR-100.

Comparison with State-of-the-art
Compared Methods: We compared our algorithm with the following methods: (1) Our baseline model which is trained on the original dataset with noisy labels. Comparing our model with this baseline enables us to clearly understand how much our algorithm has improved in performance; (2) Co-teaching [6], which trains two DNNs simultaneously. Each network teaches the other network to select possibly clean labels for training; (3) O2U-net [10], which is a lossbased method to find clean samples by cyclical training from overfitting to underfitting by adjusting the learning rate; (4) MentorNet [12], where a teacher network provides a curriculum about possibly correct samples for a student network. We compared a data-driven curriculum (MentorNet DD); (5) L q , which is a generalized cross entropy loss proposed by Zhang et al. [41]; (6) NLNL [13], where negative learning provides complementary labels and positive learning trains the model with possibly clean labels; (7) AdaCorr [42], which corrects the labels based on predictions of a noisy classifier.
Comparison Results: The accuracy of the final image classifier is provided in Table 2. The model performance is reported on the clean test set. From results we can see that cleaning the detected noisy labels and using the predicted labels significantly improves the image classification accuracy and outperforms the recent methods in all cases, especially by large margins on CIFAR-100. This suggests that it is effective for the robustness of the model to remove the most influential data which induce overfitting on the classifier.

Further Analyses of Over-Fit
Validity of Overfitting Score on the Data: To show the validity of O D , we investigated the distribution of the influences of the training samples on the validation samples. The distribution is illustrated in Figure 4, which has been obtained from the dog category in the Dog vs. Fish dataset. The horizontal axis is the index of the training data, and the vertical axis is the influence of a training sample x t i on a validation sample x v j , i.e., I D (x t i , x v j ;θ). We measured the influence on 40 validation samples for each training sample. The variation of the influence of a noisy training sample is much larger than a clean training sample, as illustrated in Figure 4. Because most influence values are distributed around zero, the variance of influences, σ k (I D (x t i , x v ;θ)) in Eq. (4) more clearly indicates the noisy-label samples than the average of the influences. This distribution appears consistently in other categories.
Noisy-label detection on Large Real-world Images: We evaluated our method on real-world image data, Cloth- ing1M. When we first trained a model for 200 epochs, the model achieved an accuracy of 64.79% as a baseline. Then, we removed the 106, 619 most influential images, which is 10% of the total training data, by running three iterations of the algorithm with α = 0, β = 0.5, γ = 11 (0.8K). When trained on new training data after removal, the accuracy largely improved to 70.03%. To confirm that the noisy labels were correctly found, the data with the highest O M and O k D from a specific label (e.g., Sweater) were checked. As expected, apparently incorrect or ambiguous images ranked at the top (see Figure 5). Therefore, we believe that our algorithm can be practically utilized when it is difficult to check the individual images among large-scale data.
Noisy-label detection on Real-world Video Data: Detecting incorrectly labeled real-world video data is significant due to the dramatic growth in popularity of video-based tasks. However, detecting noisy videos is more time-consuming than exploring images because the task requires clicking to play the videos and watching them one by one; thus, the work of noisy-label detection could benefit this area. Therefore, we extend our work to video action recognition and demonstrate that our algorithm is easily applicable to various domains.
Here, we evaluated our method on a video action recognition dataset, HMDB-51, with the TSN architecture [32]. We used a test set as a validation set and computed O k D (x t i ;θ) for 3,750 training samples. The data were checked in the order of the highest overfitting score on the data, and we unexpectedly discovered many noisy labels that were supposed to be clean. Figure 6 presents examples of detected noisy

Regularization to avoid Overfitting
As a further use case, our method can be considered as a regularization method to avoid overfitting. Deep neural networks are vulnerable to overfitting even if there is no apparent label noise in datasets. Thus, many regularization methods have been suggested to solve it [16,21,28]. Since our method can identify overfitting on individual data points, our method can be used as a regularization to smooth decision boundaries.
As a case study, we experimented on the clean CIFAR-10 data (i.e., the noise rate is 0). We used ResNet-34, setting all the training procedure the same as in section 4.1. We used 5 clean validation samples per class to estimate O D . After the 150 epochs of the training, the model achieved an accuracy of 90.8%. Then, we applied Over-Fit (Algorithm 1) on this model for three iterations with α = 0, β = 0.5, γ = 8.
As a result, a total of 1, 011 training samples were removed. When trained on a new training data after removal, the performance remarkably improved to 96.6% from 90.8%. In addition, we extracted the penultimate features, which were used to estimate the O M and O D , and visualized the features on the test dataset in Figure 7. While the original model did not separate classes well, the proposed Over-Fit could clearly separate classes as shown in the figure. We conjecture this is because we remove spurious or isolated datapoints leading the decision boundary astray, and get a well-generalized decision boundary.
(a) Original Model (b) Over-Fit is applied Figure 7. Visualization of target features using t-SNE [30]. We apply Over-Fit algorithm to the original model (a). After training on the new dataset after removing some fraction of clean samples causing over-fitting, the performance of the model has significantly improved. In addition, as it can be seen in (b), the new model can separate classes for test data better.

Conclusion
In this paper, we proposed a simple but effective noisydata detection algorithm with two novel criteria. Unlike the existing methods that aim to prevent overfitting, we interpreted the property of an overfitted model from a new perspective. We conducted various experiments to demonstrate that our algorithm can successfully detect noisy-label data. Finally, we have shown that our method can perform as a regularization technique to smooth a decision boundary. In the future, we could investigate the approaches to identify optimal hyperparameters that can be dynamically adjusted in iterations.