{'fraction_non_alphanumeric': 0.04258186228271123, 'fraction_numerical': 0.030319363967120334, 'mean_word_length': 4.56739184796199, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Learning from Mistakes based on Class Weighting with Application to Neural Architecture Search

Learning from mistakes is an effective learning approach widely used in human learning, where a learner pays greater focus on mistakes to circumvent them in the future to improve the overall learning outcomes. In this work, we aim to investigate how effectively we can leverage this exceptional learning ability to improve machine learning models. We propose a simple and effective multi-level optimization framework called learning from mistakes using class weighting (LFM-CW), inspired by mistake-driven learning to train better machine learning models. In this formulation, the primary objective is to train a model to perform effectively on target tasks by using a re-weighting technique. We learn the class weights by minimizing the validation loss of the model and re-train the model with the synthetic data from the image generator weighted by class-wise performance and real data. We apply our LFM-CW framework with differential architecture search methods on image classification datasets such as CIFAR and ImageNet, where the results show that our proposed strategy achieves lower error rate than the baselines.


Introduction
Learning from mistakes is an effective learning approach widely used in human learning. A student learns some topics from the textbook and then takes a test to evaluate how well he/she has grasped them. Topics on which the student made mistakes in the test were not thoroughly understood. The student devotes more attention on improving the understanding of these topics to prevent repeating similar mistakes in the future. Intuitively, this approach helps the student to gauge which concepts are weaker and encourages Arxiv Preprint Figure 1. Illustration of learning from mistakes. Learner takes a test after studying k topics and receives feedback on topic-wise performance. Learner revisits the topics again, focusing on weaker topics (indicated by darker shade) in comparison to other topics (indicated by lighter shade). them to concentrate more on these in order to reinforce a better understanding.
We are interested in investigating whether we can utilize this human learning method to train better machine learning models. We propose a novel machine learning framework called learning from mistakes using class weighting (LFM-CW) (as illustrated in Figure 1), an extension to Skillearn . In this framework, the formulation consists of a three-level optimization problem that involves three learning stages. In this work, we assume the end task is classification while noting that our framework can also be extended to other tasks.
In this formulation, there is a classification model and a conditional image generator (CIG). The classification model has a learnable architecture with a set of network weights. CIG is a generative adversarial network (GAN) (Goodfellow et al., 2014), where the generator has a fixed architecture with a set of learnable weights, and the discriminator has a learnable architecture and a set of learnable weights. We train a CIG, which takes a class label as an input and generates an image belonging to this class. After training an image classification model, we measure its class-wise validation performance. If the classification model does not perform in a certain class c, the CIG will generate more images belonging to c and then use these generated images to re-train the classification model. The intuition is: if the classification model does not perform well on a certain class c, we assign more weights to its training examples so as to enforce more attention of the classification model to this class.

arXiv:2112.00275v2 [cs.LG] 18 Feb 2022
A multi-level optimization framework is developed to unify the three learning stages and optimize them jointly in an end-to-end manner. Each learning stage influences other stages. We apply our method for neural architecture search in image classification tasks on CIFAR-10 (Krizhevsky, 2009), CIFAR-100 (Krizhevsky, 2009), and ImageNet (Deng et al., 2009). Experimental results demonstrate competitive performance with existing differential architecture search approaches Liang et al., 2019;Chen et al., 2019;Xu et al., 2020).
To summarize, the contribution of our work is three-fold: 1. We propose a novel machine learning approach called learning from mistakes using class weighting (LFM-CW) by leveraging the mistake-driven learning technique of humans. In our approach, we present a formulation where the model uses its intermediate network weights to make predictions and then re-train network weights by adopting a re-weighting strategy to incorporate the corrective feedback, resulting in improved learning outcomes.
2. We develop an efficient optimization algorithm to solve the LFM-CW strategy, a multi-level framework consisting of three learning stages.
3. We apply the LFM-CW strategy for neural architecture search on various benchmarks, where the results demonstrate the effectiveness of our method.

Neural Architecture Search
Neural Architecture Search (NAS) aims to automatically design and build high-performing neural architectures that can achieve the best performance on a specific task with minimal human intervention. Although existing NAS approaches use a hybrid search strategy to find an ideal candidate cell and stack multiple copies to build a larger network leveraging human expertise as seen in human-designed architectures (e.g., GoogleNet (Szegedy et al., 2015), ResNet (He et al., 2016), etc.), NAS aims to automate the architecture search procedure fully. Early NAS techniques (Zoph & Le, 2017;Baker et al., 2017;Pham et al., 2018;Zoph et al., 2018) used reinforcement learning (RL), in which a policy network learns to produce high-quality architectures by minimizing validation loss as a reward. Following RL-based approaches, evolutionary algorithms (Real et al., 2017;Liu et al., 2018b;Real et al., 2019) validated the feasibility of automatic architecture search achieving comparable results, where high-quality architectures produce offspring to replace low-quality architectures, and quality is measured using fitness scores. However, these approaches are computationally expensive and not feasible for researchers who lack sufficient computational resources. To address this issue, differentiable search methods Xie et al., 2019;Cai et al., 2019) have been proposed, which aims to accelerate the search for neural architecture by parametrizing architectures as differentiable functions and optimizing using gradient descent-based approaches. Several subsequent works such as P-DARTS , PC-DARTS (Xu et al., 2020) have further improved differential NAS algorithms. P-DARTS  progressively increases the depth of architecture during architecture search. PC-DARTS (Xu et al., 2020) samples sub-architectures from super network to eliminate redundancy in the search process. Our work is closely related to a meta-learning approach GTN  where a generative model is trained to produce synthetic examples and then uses these examples to search neural architectures. In contrast, our approach aims to search neural architecture by jointly optimizing on synthetic and real data in a single stage.

Importance Weighting
Weighting is a widely used technique in machine learning to improve robustness against training data bias. To deal with class-imbalanced datasets corresponding to distributional shifts, several sample weighting strategies (e.g., focal-loss (Lin et al., 2017), class-balanced loss (Cui et al., 2019)) have been proposed, where larger weights are assigned to difficult or easily misclassified examples and lower weights are assigned to well-classified examples. Similarly, many approaches (Axelrod et al., 2011;Foster et al., 2010;Jiang & Zhai, 2007;Moore & Lewis, 2010;Ngiam et al., 2018;Sivasankaran et al., 2017) showed that selecting or re-weighting training examples improves overall performance. Bi-level optimization-based approaches (Ren et al., 2018;Shu et al., 2019;Wang et al., 2020a;Ren et al., 2020;Wang et al., 2020c) learn data weights by minimizing validation performance of models trained using re-weighted data, in an end-to-end manner. (Liu et al., 2021) proposed a simple two-stage approach to improve group robustness, with the first stage identifying minority cases and the second stage upweighting these examples. On the other hand, our approach tries to learn network weights in the second stage using real data and synthetic data weighted based on the class-wise validation performance of the network trained in the first stage on the real data.

Problem Formulation
In this formulation (as displayed in Figure 2), there is a classification model and GAN-based CIG (e.g., CGAN (Mirza & Osindero, 2014), SAGAN , BigGAN (Brock et al., 2019), RoCGAN (Chrysos et al., 2019)) with a generator and a discriminator. The classification model has a learnable architecture A and two sets of network weights W 1 and W 2 . The generator has a fixed architecture and a set of learnable weights G. The discriminator has a learnable architecture, same as the architecture of the classification model, which is A and a set of learnable weights H. Learning consists of three stages. We summarize the key elements of this formulation in Table  1.
, where x i is an image and y i is a class label, with architecture A fixed as: Meanwhile, we train a GAN-based CIG with a generator and a discriminator. The generator takes a class name as an input and generates an image. The discriminator takes an image as input and predicts whether it is synthetic or real. The training dataset for CIG is , which is obtained by switching the order of images and labels in D (tr) cls . In this stage, when training the CIG, we train generator G and discriminator H with architecture A fixed in the following way: There are two optimization problems in this stage. In the second stage, we measure the validation performance of the where λ is a trade-off parameter, C is the total number of classes,x c,m is the m th generated image when feeding class c to the CIG model. In the third stage, we validate optimal weights W * 2 (A, W * 1 (A), G * (A)) on the validation dataset D (val) cls and learn A by minimizing the validation loss as: Putting the above pieces together, we have the following overall formulation.

Optimization Algorithm
In this section, we develop an efficient algorithm to solve three-level LFM-CW formulation strategy. Following , we approximate W * 1 (A) using one-step gradient descent of W 1 w.r.t. L(A, W 1 , D (tr) cls ) as: where ξ W1 is a learning rate for W 1 . Similarly, we approximate G * (A) and H * (A) using one-step gradient descent update of G and H w.r.t. L(G, H, A, D cig ) as: where ξ G and ξ H are learning rates for G and H respectively. We plug approximation W 1 of W * 1 (A) in Eq. (3) to obtain an approximated objective O W2 : where ξ W2 is a learning rate for W 2 . Finally, we can update the architecture A by calculating gradient of L(A, W 2 , D (val) cls ) w.r.t. A in the following way: where ξ A is a learning for A. Following chain rule, we can cls ) in Eq.(11) in the following way: The overall algorithm for solving LFM-CW formulation is in Algorithm 1 Algorithm 1 Optimization algorithm for LFM-CW formulation while not converged do Update the first set of weights W 1 using Eq.(6) Update the generator G and discriminator H using Eq. (7) and Eq. (8) Update the second set of weights W 2 using Eq.(10) Update the architecture A using Eq.(11) end while

Experiments
In this section, we apply our proposed LFM-CW strategy to image classification tasks. We incorporate it into differential architecture search approaches such as DARTS, P-DARTS, and PC-DARTS and perform experiments on CIFAR and ImageNet datasets. Following , each experiment is composed of two phases: architecture search and evaluation. In the search phase, we find out an optimal cell by minimizing the validation loss. In the evaluation phase, multiple copies of an optimal searched cell are stacked and composed into a larger network, which we train from scratch and report their performance on the test set.

Datasets
We conduct experiments on three image classification datasets: CIFAR-10 (Krizhevsky, 2009), CIFAR-100 (Krizhevsky, 2009), and ImageNet (Deng et al., 2009). Both CIFAR-10 and CIFAR-100 datasets contain 50K training images and 10K testing images, with 10 and 100 classes (the number of images in each class is equal), respectively. For each of them, we split the original 50K training images into a new 25K training set and 25K validation set. ImageNet dataset contains 1.2M training images and 50K validation images, with 1000 classes. The validation set is used as a test set for architecture evaluation. Architecture search on the 1.2M training images is computationally too expensive. To overcome this issue, following (Xu et al., 2020), we randomly sample 10% and 2.5% images from the 1.2M training images to form a new training set and validation set, respectively. We perform the architecture search on the newly obtained subset and train the large architecture composed of multiple candidate cells on the full set of 1.2M images during evaluation.

Experimental Settings
Our approach is agnostic that can be coupled with any existing differential search method. Specifically, we apply our approach to the following NAS methods: 1) DARTS  2) P-DARTS , and 3) PC-DARTS (Xu et al., 2020), where the search space is composed of building blocks such as 3×3 and 5×5 (dilated) separable convolutions, 3 × 3 max pooling, 3 × 3 average pooling, zero, and identity. We use BigGAN (Brock et al., 2019) for the CIG in the proposed LFM-CW formulation. The hyperparameter λ is set to 1. We generate synthetic images of the same target labels as the training images in a mini-batch for class-wise weighted loss described in Eq.(3) (as displayed in Figure 5 and Figure 6 in the appendix A). We report the mean and standard deviation of classification errors obtained from 10 runs with different random seeds.
During architecture search for both CIFAR-10 and CIFAR-100, each architecture consists of a stack of 8 cells, and each cell consists of 7 nodes, with the initial channel number set to 16. SGD (Stochastic Gradient Descent) optimizer was used to optimize network weights, with an initial learning rate of 0.025, a weight decay of 3e-4, and a momentum of 0.9. Cosine decay scheduler (Loshchilov & Hutter, 2017) was used for scheduling the learning rate. Adam optimizer (Kingma & Ba, 2015) was used to optimize the architecture, with a learning rate of 3e-4 and a weight decay of 1e-3. The generator and discriminator in the CIG are optimized using Adam optimizer, with a learning rate of 2e-4. The search was performed for 50 epochs in DARTS and PC-DARTS variants and 25 epochs in the P-DARTS variant, with a batch size of 64, respectively.  (Pham et al., 2018) 2.89 19.43 4.6 4.6 0.5 *AmoebaNet (Real et al., 2019) 2.55 ± 0.05 18.93 2.8 3.1 3150 *HierEvol (Liu et al., 2018b) 3.75 -15.7 -300 *GDAS (Dong & Yang, 2019) 2.93 18.38 3.4 3.4 0.2 *DropNAS (Hong et al., 2020) 2.58 ± 0.14 16.39 4.1 4.4 0.7 *ProxylessNAS (Cai et al., 2019) 2.08 -5.7 -4.0 *GTN  2.92 ± 0.06 -8.2 -0.67 *BayesNAS  2.81 ± 0.04 -3.4 -0.2 *MergeNAS (Wang et al., 2020b) 2.73 ± 0.02 -2.9 -0.2 *NoisyDARTS (Chu et al., 2020a) 2.70 ± 0.23 -3.3 -0.4 *ASAP (Noy et al., 2020) 2.68 ± 0.11 -2.5 -0.2 *SDARTS (Chen & Hsieh, 2020) 2.61 ± 0.02 -3.3 -1.3 *FairDARTS (Chu et al., 2020b) 2.54 -3.3 -0.4 *DrNAS (Chen et al., 2021) 2.54 ± 0.03 -4.0 -0.4 *R-DARTS (Zela et al., 2020) 2 During the architecture evaluation for CIFAR-10 and CIFAR-100, a larger network is composed by stacking 20 copies of the searched cell, with the initial channel number set to 36. The network is trained for 600 epochs with a batch size of 96 on a single Tesla V100. SGD optimizer is used for network weights training, with an initial learning rate of 0.025, a cosine decay scheduler, a weight decay of 3e-4, and a momentum of 0.9. For ImageNet, we evaluate two types of architectures: 1) those searched on CIFAR-10 and CIFAR-100; 2) those searched on a subset of ImageNet. In either type, we stack 14 copies of an optimally searched cell into a larger network, with the initial channel number set to 48. The network is trained for 250 epochs, with an initial learning rate of 0.5, a weight decay of 3e-5, a momentum of 0.9, and a batch size of 1024 on eight Tesla V100s. The rest of the hyperparameters for search and evaluation phase follow the same settings as those in DARTS, P-DARTS, PC-DARTS (detailed list mentioned in the appendix B). Table 2 shows the classification error (%) of different NAS methods on the CIFAR-10 and CIFAR-100 test sets, including the number of parameters (millions), and search cost (GPU days). We can make the following observations from the table. Our proposed LFM-CW framework significantly reduces the errors on the test set when compared to existing differential NAS methods such as DARTS, P-DARTS, and PC-DARTS. This highlights the efficacy of our method for searching for better architectures on a target task. For example, applying LFM-CW to DARTS on CIFAR-100 reduces the error by approximately 4% for first-order and second-order approximation. As another example, our method reduces the error by approximately 1.5% for P-DARTS and PC-DARTS on CIFAR-100. Similarly, these observations can be further extended to the architecture search on CIFAR-10, where our method achieves superior performance compared to the baselines. Overall, our method achieves superior performance on CIFAR-100 than that on CIFAR-10. This Table 3. Test error on ImageNet dataset with various NAS algorithms. Results highlighted with * are obtained from Skillearn .

Abalation Study
In this section, we perform the following ablation studies on LFM-CW formulation to understand the significance and effectiveness of different modules. For the ablation studies, we use the training and validation set for search and evaluation, same as mentioned in section 4.1.
• Ablation Setting 1. In this setting, we investigate how the trade-off parameter λ in Eq.(5) affects the classification error. We apply LFM-CW formulation to DARTS-1st for CIFAR-10 and P-DARTS for CIFAR-100. Figure  3 shows how classification error on the test sets of CIFAR-10 and CIFAR-100 vary as λ increases. We  Figure 3. How errors change as λ increases. can observe that increasing λ from 0.5 to 1 minimizes the error on both CIFAR-10 and CIFAR-100. However, further increasing λ increases the error. We conjecture that increasing λ might coerce the network to emphasize training signals from synthetic images rather than actual training images, resulting in inferior performance since generated synthetic images are of poor quality and noisy during initial learning.
• Ablation Setting 2. In this setting, the classification model with the second set of weights W 2 updates its weights by minimizing the loss on adversarially generated images only, without considering the loss on training set images. The corresponding formulation is following: During this study, our proposed LFM-CW framework is applied to DARTS-1st on CIFAR-10 and P-DARTS on CIFAR-100. Table 4 shows the classification error   16.59 ± 0.08 on CIFAR-10 and CIFAR-100 test sets for the ablation setting 2. On both datasets, we can see that combining losses from both the synthetic data and training data to train the classification network performs better than just using loss from the synthetic data in the second stage.
Loss from training data acts as a way of regularization, which does not allow the classification network to diverge from the true distribution while also leveraging noise from the synthetic data to help the classification model learn better.
• Ablation Setting 3. In this setting, we investigate how the classification error changes with the different conditional image generators. The experiment uses the following image generators in the first learning stage: DCGAN (Radford et al., 2016), WGANGP (Gulrajani et al., 2017), and BigGAN (Brock et al., 2019). On both datasets, the LFM-CW framework is applied to DARTS-1st. Table 5 demonstrates that the classification error reduces with the use of a sophisticated deep conditional image generator on both datasets. The intuitive reason is that: better image generator results in a better classification model in the second stage, which further results in better architecture, improving the overall learning.

Conclusion
In this paper, we present a simple yet effective approachlearning from mistakes using class weighting (LFM-CW), motivated by error-driven learning to search architecture for deep convolutional networks. The core idea is to enforce the neural network to pay greater attention to the classes, i.e., distributions for which it is making errors to avoid making similar errors in the future. Our proposed LFM-CW framework involves learning the class weights by minimizing the validation loss of the model and re-training the model with the synthetic data weighted by class-wise performance and real data. Experimental results on different image classification benchmarks strongly demonstrate the effectiveness of our proposed method.
An interesting future direction is to extend the current work for cross-dataset generalization similar to meta-learning by performing architecture search over a labeled dataset and extensively evaluating using supervised contrastive learning over the other unlabeled datasets in an interleaving manner. This would aid in developing effective systems in healthcare, where the human-annotated data is limited compared to unlabeled data. Another challenge is to reduce the computational resources required for our proposed strategy during the architecture search by using parameter sharing across models in different learning stages.
A. Visualization of generated images Figure 5 and Figure 6 visualizes the sample of images generated by CIG (BigGAN) in the search phase for our approach with DARTS-1st and PC-DARTS, respectively.   Table 6 and 7 show the hyperparameter settings used in different experiments in the search and evaluation phase.