{'fraction_non_alphanumeric': 0.020452310717797444, 'fraction_numerical': 0.008095706325794821, 'mean_word_length': 4.837191505643773, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using ROC and Unlabeled Data for Increasing Low-Shot Transfer Learning Classification Accuracy

One of the most important characteristics of human visual intelligence is the ability to identify unknown objects. The capability to distinguish between a substance which a human mind has no previous experience of and a familiar object, is innate to every human. In everyday life, within seconds of seeing an "unknown" object, we are able to categorize it as such without any substantial effort. Convolutional Neural Networks, regardless of how they are trained (i.e. in a conventional manner or through transfer learning) can recognize only the classes that they are trained for. When using them for classification, any candidate image will be placed in one of the available classes. We propose a low-shot classifier which can serve as the top layer to any existing CNN that the feature extractor was already trained. Using a limited amount of labeled data for the type of images which need to be specifically classified along with unlabeled data for all other images, a unique target matrix and a Receiver Operator Curve (ROC) criterion, we are able to increase identification accuracy by up to 30% for the images that do not belong to any specific classes, while retaining the ability to identify images that belong to the specific classes of interest.


Introduction
Image-label couples have been the driving force of machine learning for years. Visual recognition on the basis of extensive labeling has produced networks such as R-CNNs for detecting objects of interest in multi-object pictures [1,2], has achieved recognizing image elements using semantic segmentation [3] and even producing three dimensional reconstructions of humans [4]. Very recent efforts have been made on improving the performance of these approaches using label-inexpensive methods [5].
Transfer learning, a learning process where a new task is solved by transferring relevant knowledge from known solutions to related tasks, is a well studied method which also depends on image-label couples [6]. Nevertheless, as in all CNNs, the classifier is capable of recognizing only the classes that it was trained for. During the use of the trained classifier, any candidate image will be placed in one of the available classes (even if it is a completely different object). We challenge this expectation by introducing unlabeled training in the classifier and a statistical analysis method to create a broader recognition capability. 1: Schematic of the two parts of our network. We feed to the Pretrained Network labeled "Relevant" images and unlabeled "Irrelevant" images. For each image our proposed classifier produces probability vectors that get classified using a threshold criterion and ROC, with accuracy much greater than already existing techniques, especially for the irrelevant dataset.
Specifically, we present an approach on significantly improving the performance of a simple, time efficient, one-layer classifier on recognising labeled "Relevant" images along with non-labeled "Irrelevant" images ( Figure 1). The ability to specifically recognize a limited number (of the order of 50) of relevant classes and also identify when an image does not belong to any of them in a label-inexpensive way, is one of the main motivations for the changes to the concept transfer learning.
Transfer learning research has mostly focused on the scenario where large amounts of training data are available for novel classes. Tackling the problem of classifiers scoring low on images that have not been part of training due to the absence of labels, will greatly increase the accuracy of background recognition methods [7,8], along with creating a capability of inexpensively re-training the head of a neural network to recognize pictures considered as noise.
Existing alternatives to our method which we use to compare our results with, are classifiers trained on a basic transfer learning scheme [16]. Along with other disadvantages, these baseline capabilities require higher labeling work and they are going to be further discussed in the Results Section.
To explore the problem in a holistic, non-specific and easily applicable way we concentrate only on the training of the classification matrix, using the pre-trained feature extractors [10] discussed in Section 2. We reduce the image matrices to feature vectors [11,12] which are then used in Section 4 to train the classifier with the help of the analytic derivative of our loss function and a unique, partially labeled, target matrix. In Section 5, we use the classification variability statistics and a ROC [13,14,15] as a novel method to calculate threshold probabilities for each relevant class. By switching to a validation dataset and using the calculated threshold probabilities, in Section 6 we present the ability to effectively classify both Relevant and Irrelevant datasets. In the last Section we make some closing remarks on future work and research that can be conducted.
The resulting proposed classifier is a unique solution to generating low label and low shot learning capabilities. The simplicity of the loss function analytic derivative combined with a unique target matrix and a basic ROC statistical approach yields very good results, while taking steps forward addressing the binary classification problem in a simple and effective way. Before we start describing the steps taken towards building our own classification algorithm, we will be discussing the ConvNet parts that we have based our research on. The next section is going to introduce the feature generating algorithms we used to reduce our images to vector space, namely, the feature extractor.

Feature Extractor
To reduce the raw RGB image matrices to vectors, a process known as feature extraction, we used the algorithm written and the pre-trained weights generated by Bharath Hariharan and Ross Girshick [16]. The weights were trained on the Im-ageNet1k dataset which involves general internet imagery. We selected this simple feature extractor in order to create generic results that could be used in a variety of applications. The development of the feature extractor itself is out of the scope of this specific research. It is anticipated that potential targeted changes on the entire net and the use of a more application specific dataset would increase the accuracy even more, as using filters trained on pictures with similar characteristics to the ones we are classifying would produce more robust feature representations.
Deep Residual Networks have been proven to be a very effective in mapping images to a meaningful feature space, especially when trained in large datasets [17,18]. For time efficiency and algorithm simplicity we chose ResNet10 rather than the much deeper ResNet50. The variation of ResNet10 we use, maps the images to a vector of 512 different features, whereas ResNet50 produces four times larger feature maps, significantly slowing our testing. The different types of ResNet10 we used, although not very different, will be discussed in Section 3.
Before proceeding to the training process we normalize the features of every training image using the equation: Where is a 512 long feature map and , its maximum and minimum values in vector form. This type of basic normalization clamps the values from 0 to 1. We apply the normalization to prevent the Exponential Loss and its Derivative in Equation 9 and Equation 11 respectively from gaining extremely high values.

Datasets
To explain with more clarity our method and results, we describe the way the Caltech256 dataset is split in two subdatasets. Caltech 256 is an open source dataset, it consists of 256 different image classes and has been recently used a lot as a benchmark for transfer learning applications [24,25].
Similar to our selection of ResNet, we use an open source and broadly used dataset in order to make our example and results as general and less task specific as possible. As we intend to produce work that is going to be used in the future for specific applications, to give a hint on how the method can be geared towards recognition in unique environments, we also included a small subset of infrared (IR) images in some of our tests.
We will be trying to train our classifier on both labeled and unlabeled pictures, therefore our main dataset consists of what we call Relevant and Irrelevant pictures. The relevant group is consisted of the first 50 classes of Caltech256 and the irrelevant group of the next 50 classes as shown on Figure  2. To explore the dependency between our classifier visual recognition accuracy and the amount of unlabeled images, we created two more datasets with an expanded number of Lastly, to explore the behavior of our transfer learning method on unique and very different environments from the ones present in the Caltech256 dataset, we created our own infrared (IR) combat vehicle dataset by taking snapshots of a publicly available IR video database (examples displayed in Figure 3). The last dataset (Infrared Dataset) is composed of the same amount of pictures with the one in Figure 2, with the exception that the first 8 relevant image classes are infrared instead of Caltech 256 pictures. We use the term Infrared Dataset for it, as an indication that it contains 8 Infrared classes within the 50 relevant classes. On the next chapter we discuss how the dataset images described above are treated during the classifier training process.

Analytic Classifier Training
The two integral parts of our classifier training process are the target matrix and the loss function. Our training goal is to tweak the initially randomized weight matrix in such a manner that when multiplying it with a validation feature map, it produces a probability matrix whose largest value is the desired class element.
In machine learning, a fully connected layer performs the following calculationˆ where W is weighting matrix of the classifier, F is the feature map matrix, b is the bias vector and is the activation function. If we were to use the Singular Value Decomposition method to solve our matrix equations [26,27], the pseudo-inverse method calculation would bê Here, is the feature maps and is the weight matrix which we desire to train. is the target, the ideal outcome for the probability matrix. Our MATLAB code handles the training one class at the time, therefore is a by matrix and is a long vector for each class. With no use of the bias vector, and the reversed order of F and W to account for the row-column switch, if in SVD we calculate the W matrix one vector (class) at a time, we are essentially solving for the least square solution of When the exact solution does not exist, which means that A is not a full-rank square matrix, we get approximate solutions as Therefore the approximation error is and in a least-square approach the loss function would be substituting the approximate we get We introduce an exponential version of the least square solution in order to explore a new, faster converging loss function. Our new squared-exponential loss function is therefore the gradient can be proven analytically to be = =1 2 2 (10) and the gradient vector is There are two main reasons for choosing this loss function. A squared-exponential function is easy to differentiate analytically and the differentiation can also be applied to the linear algebra form which we use on MATLAB. Compared to other differentiable functions we tested, the squareexponential was the one to converge faster and in a steady way. A problem we encountered, which we solved by normalizing the feature maps as described in Equation 1, is that because of the nature of the function, for numbers greater than 1, the Loss would get extremely high values.
The gradient matrix in Equation 11 is then multiplied (MATLAB notation for element-wise multiplication) by a learning rate and added to the weight matrix, repeating this sequence for every epoch. The steps taken towards training the classifier matrix are therefore all independent from MATLAB machine learning libraries or functions. Here we need to note that our implementation from the very beginning was geared towards MATLAB. Although many different loss functions are being used by machine learning libraries, where they get differentiated in a semi-analytic fashion, we concentrated our efforts on not using any existing libraries. Therefore the squared-exponential loss function was proven to be a good fit.
As in most machine learning applications, the update mechanism we use towards convergence is some variation of a normal gradient descent equation. In our specific case we use Here, in every epoch k, W gets updated by subtracting from it the product of the learning rate and the gradient matrix. We obtain our learning rate using an algorithm inspired by ISTA [28]. We begin with calculating a pseudoloss which is going to be compared with the actual loss to determine whether the learning rate needs to be decreased or kept as specified on the previous epoch. This iterative method progressively decreases the learning rate as we approach closer to the desired optimal point.
The last, and most unique part about our classification method is our target. As mentioned in the Section 1, the uniqueness of our approach relies on the fact that we make use of unlabeled images during the training of the weight matrix. This is done by extending a typical one-hot encoding [23] matrix to include even probability distributions as the targets for the Irrelevant images. Labeled images have arrays of zeros and a singular value on the correct class element as targets. We intend to "label" the irrelevant images by assigning to them targets where all elements have the same value.
Irrelevant pictures belong to none of the classes therefore the probability for each class should be zero, or 1/ if we want the probabilities to add up to 1. By experimentation we concluded that in the case of our specific application and dataset, the irrelevant target that works best should be a slight negative value, such as -0.2. This intuition matches some of the binary classification work that has been done on Support Vector Machines' (SVM) correlation filters, where 0 and 1 were not as separable as a negative value (-0.1, -1) and 1 [19,20]. As an example, if a training dataset was consisted of six pictures, half of them labeled and half of them unlabeled, and the labeled ones were members of three different classes, our target matrix would look as follows: We train the weight matrix in such a way that during evaluation the irrelevant images probability vector values are spread equally between the classes. This helps the Irrelevant pictures to score less than the respective class threshold.
A future research idea discussed in Section 7 would adjust the Irrelevant target numbers differently for every picture depending on its statistical profile. Along with using this target oriented training procedure, we also increase our recognition accuracy by calculating our threshold probabilities using the ROC method. In the case it is not possible to obtain a significant amount of statistical knowledge about the unlabeled data, a method similar to Transductive SVMs could be used to leverage information from irrelevant pictures [21,22].

ROC Threshold Calculation
The multiplication between a feature vector and a weight matrix yields a probability vector. Neural Network classification theory uses the highest probability (Top 1, Top 3 or Top 5 have been used too) to group the images into classes. We extend this criterion to make it applicable when unlabeled images are present by introducing a Threshold Probability ( ) value for each class.
The value serves as a binary discriminating test in order to group pictures in the Relevant and Irrelevant bins. As mentioned above, we don't only need to divide pictures in the two groups, we also want the labeled group pictures to be normally classified in their respective class.
It is important to note that the is calculated by classifying the training images dataset. We need the to be pre-calculated before we start evaluating our validation dataset. Once the is known the classification process runs as follows: a) The validation image runs through the classifier and probabilities, which denote the likelihood of the image belonging to each class, are calculated. b) The image is classified as belonging to the highest probability class. c) The threshold probability criterion is applied within each class in order to group out the irrelevant images.
Within Figure 4 we present an example where probability values for images from two classes are plotted. During the training of the classifier matrix we treated the Chess Board pictures as labeled (Relevant), with label 45 attributed to them, and the Grapes pictures as unlabeled (Irrelevant). On the graph we can see that during validation, most of the Chess Board pictures have high probability values on the correct class (45). The correct probabilities are also above the of this class, therefore they will not be classified incorrectly as Irrelevant.

F 4:
In blue and red we see the 50 different class probability values of the 10 different Chess Board and Grapes images respectively. In Green is the ROC threshold calculated for each one of the 50 classes. We observe that for a very limited amount of probabilities we score above the at the wrong class (not 45), therefore this is our classification error.
On the other hand, the Grapes pictures which are treated as Irrelevant are being observed to have lower probabilities on average compared to the relevant ones, but also lower than the respective . Our target matrix along with our classification method achieves to push the Irrelevant probabilities lower than the Relevant, while keeping the correct Relevant probabilities over the Threshold. Intuitively, we can set the to be the lowest relevant value. If a picture is not classified higher than the worst correctly classified training picture, then it should be Irrelevant. As seen in Figure 5, although this discriminatory rule will give us the best possible Relevant accuracy, it will strongly discriminate against Irrelevant pictures. To maximize the combined accuracy we use the Receiver Operating Characteristic method [13,14,15] to chose our threshold values. The same way we would do with a Normal Threshold, the ROC Threshold is going to be calculated right after training and before validation, using explicitly the training data.
To demonstrate the need of using the ROC , we graph the ROC curves of five, unique compared to each other, classes of pictures that we trained our classifier on. All five classes were part of the labeled dataset. We can see in Figure  6 that every different class of pictures has a different response to the ROC implementation. The different Areas Under the Curves (AUC) represent how well our ROC method achieves to classify the data, but also underlines the need of such an implementation.
In our analysis we focus on the classification of Relevant pictures, which we assume to be the Positive statistical case, therefore we use the terms True Relevant Rate (TRR) and False Relevant Rate (FRR). TRR and FRR are no different than the True Positive Rate (or Sensitivity) and False Positive Rate (or Fall-Out) respectively, used in statistical analysis. Therefore we define: In Figure 6 it is obvious that the IR Human class is so unique that the classifier does not have any trouble distinguishing it from the rest of the dataset, therefore as seen in the figure above its AUC equals to 1 and the ROC does not have much effect on its cumulative accuracy. Different classes though present different levels of difficulty for our classifier. The Chimp class as seen above, has an AUC of 0.91, which means that the ROC can significantly improve its cumulative accuracy if a is picked wisely. The Normal Threshold would pick the point on the graph where the False Relevant Rate is minimum for a True Relevant Rate of 1, hence, for the Chimp graph, the (0.57, 1.00) point. Using our ROC algorithm we can pick any other point on the graph, such as the (0.20, 0.84) point which is the one further away from the blue line that represents a random guess. By doing this, although we slightly decreased our TRR, we get a great increase in FRR, which results to a significantly higher cumulative accuracy. This accuracy increase is going to be clearly presented and discussed in the next Section.

Testing & Results
At the last section of this paper we are presenting the effectiveness of the new method in classifying Relevant images and determining when an image is Irrelevant. In order to demonstrate the new development, we created as a point of reference two baseline results using the TL approach and a typical single layer classifier.
The first baseline result (Only Labeled TL), was produced by using transfer learning [16] and training the classifier only on labeled images of the Relevant classes. This is the case where although we have unlabeled images for the Irrelevant classes, we do not use them, expecting the labeled images to have enough meaningful features to accommodate recognizing the Irrelevant ones. To evaluate this method, we use our Normal Threshold Probability Criterion discussed above where we set the lowest correct Relevant training probability as the threshold for each class. During testing, if the image's highest probability scores higher than the respective threshold then its classified as Relevant, if not as Irrelevant.
The second baseline result, which we call "+1 Class", was generated by using transfer learning [16] and training the classifier to recognize the Relevant classes along with one extra class which encapsulates all Irrelevant images. During the training of the classifier, all unlabeled images of the Irrelevant classes were assigned to an extra class. The evaluation is being done by simply comparing the highest scoring index of every image with the correct target. Table 1 shows how the baseline methods scored for both Relevant and Irrelevant images compared to Our Method, with and without applying the ROC optimization. As "Normal" we describe the dataset consisted of 50 Relevant and 50 Irrelevant Caltech256 classes and "Infrared" is the dataset where we have substituted 8 of the Relevant classes with IR ones. Both datasets are described in Section 3. The "Our Method" results are obtained by running the algorithm described up until Section 6 and the "Our Method w/ ROC" by adding the ROC extension. "R" and "I" are the Relevant and Irrelevant classification accuracy respectively. The numbers shown in the tables are the percentages of images that got classified correctly during evaluation.
It can be observed that the baseline methods are unable to classify decently both groups of images. The "+1 Class" method seems to over-train the classifier on recognising the unlabeled images failing to put the labeled ones in the correct classes. This happens most likely due to the unbalanced training data, as the 51st class has as many images as the rest 50 together. On the other hand, by using only labeled images, we train the classifier to specifically recognise the labeled group, failing to filter out the unlabeled images "noise".
In the first row of Table 1, the results of our classifier without the ROC extension show that our loss function combined with our unique target matrix and the threshold criterion can recognize equally well both labeled and unlabeled images. It is notable that for the Relevant group our method loses an insignificant amount of accuracy compared to the label-specific baseline method.

F 7:
The legend follows the color coding of the four different methods in Table 1. For each of the four methods the graph has four different sub color groups with three data points each. The four different sub groups represent the different datasets discussed in Section 2 and the data points are the three different pre-trained feature extractors used.
The ROC method greatly increases the unlabeled images recognition, to the expense of the labeled images. This is not a problem as we will show how it is able to tweak the ROC in such a manner that it does not heavily discriminate against one group. The table shows the importance of using the ROC to greatly increase the cumulative accuracy. Our ROC classifier increases by 12% the cumulative recognition scores compared to the "+1 Class" method and by 25.4% compared to the label exclusive transfer learning method. The Table 1 results show us that unlabeled images can be used in a meaningful way on transfer learning applications.
The described results are also depicted in the Accuracy Comparison Graph in Figure 7. For every method discussed we use three different ResNet10 feature extractors (BatchSGM, SGM, L2) in order to show the consistency of our classifier results.
With a few exceptions, no matter the feature extractor or the nature of our dataset (including infrared, including more unlabeled images), our proposed method not only provides a higher cumulative accuracy but also eliminates the bias between labeled and unlabeled images by classifying both equally well when compared to the baseline approaches. In the graph we introduced the results of our extended datasets which consists of more unlabeled images. Table 2 offers a closer look to the comparison of the two extended datasets. We follow the same notation in Table 2 as used in Table  1, with the only difference being that the "+50" and "+150"  Table 2 and the plot in Figure 7, that our method still scores better in a cumulative perspective, we can see that biases against the Relevant (in Classifier w\ ROC algorithm) or Irrelevant (in Classifier algorithm) group begin to occur when increasing the amount of unlabeled images.
We see that the more we increase the Irrelevant to Relevant ratio the worse we score on the Irrelevant part. This might seem counter intuitive as we would expect the more unlabeled images we feed, the better we are able to recognise them. In reality, we introduce many more feature elements on the irrelevant part, which leads to consequently eliminating their uniqueness. This observation suggests how well our classifier might work on recognising background noise from actual pictures of interest in the case of Semantic Segmentation [3].
When introducing the proposed ROC approach on the second row we can still observe the introduction of bias, especially on the 150 Class Extended Dataset. Yet again, this problem could be mitigated by tweaking our ROC in such a way that it is non discriminating against Relevant pictures. On the last table we present the results of our tweaked ROC Classifier being used on the +150 Irrelevant dataset. The same behavior is observed when we test the +50 Irrelevant dataset. The "Only Labeled TL" and "Optimal ROC" rows correspond to Table 2 second and fourth data rows. Putting a constraint on how much we are willing to shift the to limit the loss in Relevant, affects negatively the Irrelevant. We desire to find a percentage which during testing gives us a decent cumulative accuracy without big losses on the Relevant part. This could be imagined as turning a knob to tune our ROC implementation. This can be specific in every application, therefore an open ended approach is adopted.
An 100% constraint would be Our Method without ROC, as we set our Threshold Probabilities to be the lowest correctly classified Irrelevant picture in every class. On the table presented, a 90% Constraint means that we ask our ROC algorithm to keep our thresholds to a value that will not hurt our correct Relevant guesses more than 10% during the calculation of the . Therefore these constraints are applied when using the training images and they differ from the percentages encountered in the validation ( Table 3). As we can see, for the specific case we can compromise with an 18.5% total increase instead of the the 21.5% of the optimal case, in order to get a more equal recognition accuracy.
The proposed ROC not only makes the resulting model more flexible and easy to customize depending on the needs of the datasets, but also makes the method fit for any application. This is a specifically interesting feature of our work, as we can use the classifier as an extension to any image recognition algorithm which desires to filter out Irrelevant images without the expense of labeling. This brings us back to the importance of the work that needs to be done in the future.

Future Work
Throughout the duration of this research project we have been using MATLAB to train and test our one layer-network. A Python implementation using libraries such as TensorFlow or PyTorch to take advantage of GPU computational power would be beneficial both for the researcher and the algorithm itself. As mentioned in the introduction, speed is one of the aspects of the algorithm that makes it very versatile.
Throughout the paper we have been discussing how our examples and datasets consist of a wide variety of images that are not linked to each other outside the class groups. Applying the abstract and rather general method we constructed in a more specific dataset or targeted example is expected to yield even better results. The first step towards this direction was made by introducing infrared army vehicle images in the relevant dataset.
An idea briefly discussed on Section 4 would be to use a clustering method (such as k-means) to group the unlabeled images and assign pseudo-labels to them. Consequently, we will be able to identify which groups are statistically similar to the labeled classes and therefore most likely to be classified as such. This way, in a target training manner we will be able to adjust the unlabeled targets (which currently are all -0.2) to a value that will produce a more even probability distribution for each Irrelevant picture. Such a method would greatly increase not only the Irrelevant accuracy but also the effect ROC has on the .
Lastly, a more time consuming and difficult task would be to further researching which feature extractors are more appropriate to have our classifier "worn" on, and using what types of feature extractors would yield better results for the aforementioned specific examples.
The code we based our experimentation and results on is available at: https://github.com/skasapis/ROCUnlabeledClassification