{'fraction_non_alphanumeric': 0.03233453703448466, 'fraction_numerical': 0.026159825784932603, 'mean_word_length': 4.7822760599298695, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
AI outperformed every dermatologist: Improved dermoscopic melanoma diagnosis through customizing batch logic and loss function in an optimized Deep CNN architecture

Melanoma, one of most dangerous types of skin cancer, re-sults in a very high mortality rate. Early detection and resection are two key points for a successful cure. Recent research has used artificial intelligence to classify melanoma and nevus and to compare the assessment of these algorithms to that of dermatologists. However, an imbalance of sensitivity and specificity measures affected the performance of existing models. This study proposes a method using deep convolutional neural networks aiming to detect melanoma as a binary classification problem. It involves 3 key features, namely customized batch logic, customized loss function and reformed fully connected layers. The training dataset is kept up to date including 17,302 images of melanoma and nevus; this is the largest dataset by far. The model performance is compared to that of 157 dermatologists from 12 university hospitals in Germany based on MClass-D dataset. The model outperformed all 157 dermatologists and achieved state-of-the-art performance with AUC at 94.4% with sensitivity of 85.0% and specificity of 95.0% using a prediction threshold of 0.5 on the MClass-D dataset of 100 dermoscopic images. Moreover, a threshold of 0.40858 showed the most balanced measure compared to other researches, and is promisingly application to medical diagnosis, with sensitivity of 90.0% and specificity of 93.8%.


Introduction
Skin cancer is one of the most frequent cancers in the world [1]. Approximately 5 million new cases are detected each year in the USA alone. Besides, melanoma is the skin cancer with the highest death rate. There are almost 60,000 deaths in total over 350,000 melanoma cases in 2015. Despite this highest mortality rate, melanoma can be cured in up to 95% of the cases if the cancer is detected in its early stages [2]. Frequently, skin cancer can be detected by a dermatologist by using visual examination of skin lesion images and then pathological analysis if there is a suspicion. Automated binary melanoma classification using skin lesion images inspires the development of adapted techniques from artificial intelligence-based computer vision [2]. Recently, deep convolutional neural networks (CNN) have achieved very excellent results in image recognition, and exceed human accuracy in some problems with large datasets [3]. Many recent researches have used Deep CNN for the melanoma classification problem [4]- [8] but there are still open challenges due to the problems of limited data and data imbalance.
Dermoscopy is an imaging technology used to eliminate the surface reflection of skin. The visualization of deeper levels of skin lesion is enhanced when the surface reflection is eliminated. Many researches have demonstrated that, when used by expert dermatologists, this technique produces high diagnostic accuracy, when compared to standard photography [9]. In near future, cheap dermatoscope devices will be available in the market to operate on smartphones, and the chance for automated dermatological diagnostic algorithms is due to positively influence healthcare and thanks to generalized early detection. Recently, ISIC 2019 [2] has released the latest and largest dataset including 25,331 dermoscopic images of 8 different categories. This large data set helps solve the problem of insufficiency of labeled data for training Deep CNN, promising to increase the accuracy of the algorithm. In addition, Titus J. Brinker et al. [8] released the MClass-D dataset of 100 dermoscopic images together with the diagnosis results of 157 dermatologists from 12 universities in Germany. This makes it possible for researchers to use the public data set to train models, and then compare the performance of artificial intelligence models with physicians based on this dataset.
Recently, some researchers [4], [7], [10]- [13] used artificial intelligence to classify melanoma vs nevus and compare their algorithms performances with that of dermatologists. In 2017, Esteva et al were the first to compare the direct performance of the Deep CNN with that of 21 board-certified dermatologists on 111 (71 malignant, 40 benign) images dataset, achieving AUC of 91% [4]. In 2018, the best-performing fusion algorithm of twenty-five teams of the 2016 International Skin Imaging Collaboration ISBI Challenge was compared with that of eight dermatologists on a dataset of 100 images. The best algorithm achieved greater AUC than dermatologists (86% vs 71%) and specificity of 76% at sensitivity of 85% [10]. Tschandl et al's research compared performance of Deep CNN with that of 95 dermatologists (including 62 boardcertified dermatologists) on a test dataset of 2,072 cases in 2019. The corresponding performance of CNN was an AUC of 74.2% and sensitivity of 80.5% at specificity fixed at 51.3%. It was better than human performance which reached an AUC of 69.5% and sensitivity of 77.6% at same specificity [7]. For the first time Haenssle et al. [14] compared a CNN's diagnostic performance with an international group of only 58 dermatologists on a test dataset of 100 images. Their solution outperformed most dermatologists. In 2019, the Deep CNN system proposed by Brinker et al outperformed 136 of 157 dermatologists from 12 university hospitals in Germany on 100 dermoscopic images of MClass-D. At a mean sensitivity of 74.1%, it achieved higher specificity than the mean of dermatologists (86.5% vs 60%). Moreover, the gap of between sensitivity and specificity is still very high. In 2018, Pham et al's solution achieved AUC of 89.2% with high specificity of 97.1% and very low sensitivity of 55.6% [5]. The solution of Gessert et al [13], the winner of ISIC Challenge 2019, performed AUC of 92.8% with specificity of 96.2% and low sensitivity of 59.4%. M. Q. Khan et al [15] used image processing techniques to extract features then classify melanoma by Support Vector Machine. Their method was trained and evaluated on a small dataset of only 397 images. A. A. Adegun et al's deep learning-based solution [16] employed multi-stage and multi-scale approach. It relied on a softmax classifier for pixel-wise melanoma classification and was evaluated with two small skin lesion benchmarks (ISIC 2017 and PH2). ISIC 2017 has 2000 training images and 600 testing images while PH2 has 200 training and 60 testing images. Therefore, the problems are: 1) most studies were compared with limited numbers of dermatologists and no study has outperformed all individual experts when a big number of dermatologists was involved [4], [11], [14], 2) the sensitivity and specificity measures are imbalanced, and 3) a risk of underfitting.
Thus, in this study, we propose a solution that uses a CNN architecture in combination with a customization of batch logic, loss function and fully connected layers for binary melanoma classification. Firstly, we changed the CNN architecture from In-ceptionV3 to DenseNet169 combined with redesigned fully connected layers and an increased number of layers to solve underfiting problems and we used batch normalization and dropout to avoid overfitting. Secondly, we used the latest and largest dataset of 17,302 dermoscopic images of melanoma and nevus to train our model and then select best model by minimal loss of validation. Finally, we compared our solution performances with that of 157 dermatologists from 12 university hospitals in Germany based on same MClass-D dataset. The result is that our proposed solution outperformed all 157 dermatologists and achieved state-of-the-art (SOTA) AUC (area under the curve) at 94.4% for a high sensitivity of 85.0% and specificity of 95.0% with prediction threshold of 0.5 and at sensitivity of 90.0% and specificity of 93.8% with prediction threshold of 0.40858 on the MClass-D dataset of 100 dermoscopic images. Besides, our research details the performance of three important alternatives: 1) the use of classic batch logic and of a loss function for CNN, 2) only customized batch logic and 3) both customized batch logic and loss function. All three cases use fully customized connected layers. An important goal of this research is to design a Deep CNN architecture more effective than state-of-the-art binary melanoma classification approaches, including the direct assessment by dermatologists. Thus, we used the MClass-D dataset of Titus J. Brinker et al. [8] to evaluate our model. This dataset contains 100 dermoscopic images (20 melanoma images, 80 nevus images) and enables researchers to compare their algorithm performances with that of 157 dermatologists.  Out of the participants, 151 (96.2%) physicians are practicing in hospitals and 6 (3.8%) are dermatologic resident physicians working in a private office. Besides, there are two groups of junior physicians and board certified, accounting for 56.1% and 43.9% respectively. In terms of experience, 52.9% have experience less than or equal to four years and 47.1% have over four years of experience. Moreover, in these 157 dermatologists, 3 people had 100% sensitivity with highest specificity of 45%, and 68 people had sensitivity of at least 80% with highest specificity of 72.5%.
Ethical approval: Heidelberg University's ethical committee has waived the need for ethical approval because all dermatologists who voluntarily participate in reader study were anonymous and training of artificial intelligence algorithms was done with open source images.

Proposed classification system
In this research, we proposed a melanoma classification system which includes three main components: Batch logic, CNN and Custom fully connected layers as shown in Figure 1. In our research, the specifics consist of new steps such as: 1) customizing the logical batch to change the proportions between majority and minority classes in each batch when training the model, 2) selecting suitable CNN architecture for the melanoma classification and 3) designing new fully connected layers to solve underfiting and avoid overfitting problems. More details of the components are explained in section 3.2.

Component Details
Our research focuses on designing new deep learning architecture using the popular CNN model combined with the customization of loss function and fully connected layers for binary melanoma classification. After the evaluation of possible solutions, the most optimal model obtained has the following main characteristics: CNN: In our system, the popular CNN architecture is used for feature extraction. We investigated outstanding CNN architectures, such as InceptionV3 [17], ResNet50 [18], DenseNet121 and DenseNet169 [19], with classic batch logic and loss function to determine a network without underfitting problem in training process. ResNet50, DenseNet121 and DenseNet169 architectures result in good performance when training but DenseNet169 achieves the best performance on validation and Test-10 dataset. Thus, the DenseNet169 architecture is used to conduct the next step of customized batch logic, customize loss function, and customized fully connected layers in our research. The DenseNet169 network is transferred learning from the network trained by ImageNet, and the features are taken from its last layer named ReLU.

FIGURE 2. Dense connections in DenseNet
We experimented with InceptionV3, ResNet and DenseNet to select the appropriate architecture for melanoma classification. However, the problem of InceptionV3 is underfitting; InceptionV3 does not always converge during training. As for ResNet, the training process converged, but its sensitivity, specificity has a large deviation and AUC is low on Test-10 and MClass-D. DenseNet169 obtains the best result of all architectures on the three measures of sensitivity, specificity and AUC. This is because of the key differences between DenseNet and the other 2 models. As shown in Figure 2, in DenseNet the key idea is 1) to use the input of each layer to later be part of its output and 2) instead of using additional functions like ResNet does, DenseNet uses a concatenation function to calculate output from x and f(x). These two features explain why this approach is adequate for image classification with images containing few and large objects, as is the case in the binary melanoma classification task.
Fully connected layers: while researching the effects of loss functions on melanoma classification using the InceptionV3 architecture, it turns out that the highest sensitivity on the training set is 0.88 and roughly 0.75 on the validation set. We considered this result as an underfitting issue. Thus, our study focuses mainly on optimizing fully connected layers to avoid underfitting. This is commonly caused by a network with a simple architecture. We customized the architecture to have two hidden layers with 1,024 nodes at the first hidden layer and 512 nodes at the second hidden layer with ReLU activation instead of one hidden layer of 521 nodes with ReLU activation. After each hidden layer, we use a dropout block with rate of 0.5 to avoid overfitting. The output layer of one node with sigmoid activation was applied for our binary classification.
Batch normalization: to avoid overfitting, we applied batch normalization [20] at the two hidden layers prior to the activation function. This resulted in much higher effectiveness than non-batch processing or applying only at the first hidden layer.
Loss function: A fundamental way of training a deep convolutional neural network is by optimizing node weights to improve the model so that its output fits the ground truth data as much as possible. This is mainly executed by using a loss function. Common loss functions used in research are: mean squared error, binary cross entropy, squared hinge, sparse categorical cross entropy, categorical cross entropy, categorical hinge, etc. These loss functions work effectively on some public datasets, such as MNIST, CIFAR-10 and Imagenet because those tasks are evaluated through accuracy, the rate of correct classifications. However, applying these functions to binary melanoma classification has not proven effective because the performance evaluation criteria of this problem are composed of three indicators: AUC, sensitivity, and specificity. There is a problem of low sensitivity versus specificity. This is because during training the Deep CNN will optimize the weight of the network nodes to minimize the loss on all data of each batch train, thereby increasing the performance of accuracy. In melanoma classification, accuracy does not represent system performance due to the problem of imbalanced datasets [21]. Thus, we need to optimize sensitivity and specificity separately. In this study, we use the custom loss function (CLF), which is calculated based on the separate loss of the majority and minority classes to balance these two indices and improve AUC. CLF is calculated by positive mean squared error (PMSE) which is separately calculated on the minority class (melanoma class) and negative mean squared error (NMSE) calculated on the majority class (nevus class). If there is a set of M samples including N samples in the negative class and P samples in the positive class and and * are the truth value and the predicted value of sample ℎ respectively, then the mean squared error (MSE), PMSE, NMSE and CLF can be noted as below: In this study, CLF is constructed with a = 1.0, b = 2.0 and c = 2.0. The difference between MSE and CLF is that CLF is calculated based on the individual error of the majority class (nevus class), minority class (melanoma class) and adds the difference between the two errors. CLF increases the weight for the difference to make the penalty worse when the model has too much difference between these two classes, thereby helping to balance out sensitivity and specificity. In Table 3, we demonstrate three examples of melanoma confusion matrix on a 100-image dataset, 20 images are positive (minority class) and 80 images are negative (majority class) similar to the MClass-D ratio. One can observe that the ACCs of all three examples decrease from example 1 to example 3, but the balance between sensitivity and specificity increases. To express this problem through loss, we use mean squared error, and our proposed customization loss function. Example 3 is what we aim at, even though ACC is lower, because sensitivity and specificity are close, hence the loss needs to be minimal. In the contrary, example 1 needs the largest loss. It is also evident in the three examples in Table 3 that MSE does not reflect this as well as CLF, since MSE increases from example 1 to 3, while CLF decreases and better reflects the requirements of the problem. Optimizer: The selection of the optimizer for the network is very important and influences the process of optimizing the weight of the nodes and the performance of the model. In this study, after building the desired network architecture, we experimented with some optimizers such as Adam and SGD [22] then found that the Adam optimizer works better. Thus, we use Adam optimizer for deep experiments. Regarding the initialization of parameters for the optimizer, we use the following settings: lr= 0.0001, beta_1=0.9, beta_2=0.999, decay=0.0, epsilon=None and amsgrad=False. The choice of lr will be explained below, while beta_1, beta_2 are not zero to avoid local minima.
Learning rate: The learning rate (lr) value greatly affects the training of DCNN. Usually, when the lr is high in the first epochs, it then decreases in the later epochs, and the model fastly achieves high performance. To do this automatically, we initialize the learning rate for the Adam optimizer to 0.0001, then we use the cyclical learning rate (CLR) [23] to automatically change the lr in the range from base_lr = 0.0000001 to max_lr = 0.0001 to achieve improved classification performance without the need to tune and to converge in fewer iterations. Besides, we set mode = trian-gular2 to halve max_lr after each cycle step_size = 4 epochs as depicted in Figure 3. Batch logic: in the training process, the CNN network breaks down the large training dataset into smaller subsets by batch logic to avoid complex computation of large numbers of samples. The number of samples in each batch is batch_size and its elements are randomly picked. This leads to variations in the class distribution of each batch, notably compared to their distribution in the entire training set. In this study, we changed the logical batch so that the ratio is always constant and equal to the ratio between the two classes in the training dataset. This is to maximize the effect of the loss function on the imbalance problem.

Experimental results
In this study, to evaluate the effectiveness of the proposed system, we train the network with the training dataset and select the final model with minimal loss on the validation dataset. Then we evaluate the performance over the Test-10 dataset and compare it to previous work and to the results of 157 dermatologists on MClass-D dataset [8]. Our study proposes a new deep architecture with novelty in the introduction of the three following components: batch logic, loss function and fully connected layers. The study tested and analyzed the results with 3 scenarios as follows: 1) unchanged batch logic and loss function (ORI), 2) changed the batch logic as described above, with no changes with respect to the loss function unchanged (BON) and 3) changed both batch logic and loss function (BLF). All three cases include fully connected network layers as shown in Figure 1. After building deep architectures, we train the network with the same number of epochs of 50 and batch_size of 32. The reason for chosing 50 epochs is simply that when conducting experiments, we found that with a lower number epoch (close to 30), the sensitivity and specificity of all three models reached a maximum number of 1. We thus increased the number of epochs by 20 to keep the weights of the network stable. During the training process, we saved the model after every epoch with the smallest val_loss, and after completing 50 epochs, we used the model with smallest val_loss for evaluation. After conducting experiments, we analyzed the results with the following content: 1) the trend of SEN, SPE, and LOSS during training, 2) the performance over the Test-10 dataset and 3) the performance in comparison to that of the 157 dermatologists on the MClass-D set. In this of binary classification, the value returned by the classifier is a real number within [0, 1]. Depending on thresholds, the binary value is either assigned to 0 (negative or nevus) or 1 (positive or melanoma). We can then calculate sensitivity and specificity as in Formula (5). The combination of sensitivity and specificity is measured through a graph with specificity on the horizontal axis and sensitivity on the vertical axis, allowing to draw a corresponding ROC curve as shown in Figure 6 and 8 with the AUC being defined as the area under the curve. AUC is also represented by the percentage of the area that is under the curve, thus with a value ranging from 0% to 100%.

Evaluation of loss, sensitivity and specificity during training and validation
In this study, the changes in batch logic, loss, and fully connected layers aim to optimize the training process. The goal is to increase sensitivity during training and over the validation dataset when the loss is reduced. Figure 4 shows the change of loss, sensitivity, specificity during training (train_loss, train_sen, and train_spe) and on the validation dataset (val_loss, val_sen, and val_spe).

Loss of train and validation:
The objective of the training is to establish weights for the nodes of the network so that the train_loss value is as small as possible. In this study, the loss function of ORI and BON uses the MSE function and has same unit because they use the same batch size; we therefore can compare both values and trends. In contrast, the loss function of BLF is customized and uses a different unit than MSE, therefore only trends are comparable. As shown in Figure 4 during training, all three models of ORI, BON and BLF have substantial different loss with the first 30 epochs. Their stability and convergence reach a minimum value close to 0 from epoch 30. However, only val_loss of BLF is stable and has a decreasing trend from epoch 30 to 50. On the contrary, BON is unstable at from epoch 35 to 50. ORI's val_loss is always smaller than that of BON, however the value faces an increasing trend from epoch 40 to 50. This shows that in the process of training, val_loss of BLF is the best, while BON's val_loss is the worst. This also means that MSE loss function (ORI, BON models) is not adequate, but that our CLF loss function (BLF model) is suitable to train our Deep CNN with melanoma data.
Sensitivity and specificity: the ultimate aim of reducing train_loss is also to increase train_sen and train_spe. As the results show in Figure 4, during training, along with the change of loss from 1 to 30 and 30 to 50, sensitivity and specificity of ORI, BON and BLF models have substantial fluctuations when from epoch 1 to 30, then stable and close to 1 from epoch 30 to 50. However, there are differences in val_sen and val_spe. While val_spe of BON is the highest in every epoch, that of BLF is always the smallest. In contrast, val_sen of BLF is always the highest one and that of BON is the lowest in all three models. This suggests that the BLF model has the best balance while the BON model has the poorest one during the training process and on the validation dataset.
By analyzing trends in loss, sensitivity and specificity, it can be concluded that the BLF model is the most efficient and most balanced, the ORI model ranks second, and the BON model is the worst.

Evaluation performance over the Test-10 dataset
After selecting the best model based on training and validation datasets, we use the Test-10 set (its test data set being composed of 10% of ISIC 2019) to assess the model. The assessment is based on 1) AUC, sensitivity (SEN), specificity (SPE) with threshold = 0.5 and 2) the receiver operating characteristic (ROC). Figure 5 and Figure 6 describe these aspects respectively.
Performance of the three best models evaluated over Test-10 dataset, using a prediction threshold of 0.5: Figure 5 shows the performance of AUC, SEN and SPE of the best models of ORI, BON, and BLF. Figure 5 shows that the AUC of all three models is similarly high, the highest being BON with AUC of 93.9 while the lowest is BLF although only 0.3 percentage points lower than BON. However, when considering the balance of sensitivity and specificity, BLF is the most balanced with respec-tive sensitivity and specificity scores of 81.3% and 92.3% (difference of 9%), while BON is most unbalanced with 73.6% and 95.4% (difference of 21.8%), and ORI scores 77.6% and 92.9% (difference of 15.3%). BLF's specificity is only lower than that of ORI by 0.6 percentage points while its sensitivity is 4.7 percentage points higher. This demonstrates the effectiveness of BLF in maintaining high AUC and specificity while balancing sensitivity and specificity.

FIGURE 5.
Performance evaluated of three best models of ORI, BON, and BLF over Test-10 dataset using a prediction threshold of 0.5.
Performance evaluated of three best models over Test-10 dataset using ROC curve: Figure 6 illustrates the ROC of the three best models. Three lines in colors blue, green and purple are respectively representing the ROC curves of ORI, BON and BLF. In general, the three lines are similar and there are not many differences across the points, reflecting the fact that the differences in AUC are negligible. However, it can be seen that the performance over the Test-10 dataset is higher than the performance of all doctors (with red dots) on the MClass-D dataset excepted two doctors. In addition, BLF shows exceptional performance when specificity is greater 80% or when sensitivity is between 70% and 90%. Further detailed analysis of thresholds is provided in Table 4.  Table 4 shows that with Test-10, BLF proves the most effective of the three models when sensitivity (SEN) is between 74.1% and 90%. All three models have SEN and specificity (SPE) balanced when SEN is at 85.0% with corresponding SPE of ORI, BON and BLF respectively at 88.6%, 88.8% and 89.9%. At this level of SEN, the BLF model remains the best-performing model with SPE of 89.9%, 1.3 percentage point better than the worst model of ORI. ORI and BON perform similarly with a difference of only 0.2 percentage points.

Performance comparison with dermatologists
In this study, to compare the performance of AI models with that of dermatologist, the best three models with smallest val_loss are evaluated on the MClass-D dataset, the one used to evaluate 157 dermatologists as described in the related work. In the present section, we evaluate the work presented through 1) evaluating performance over MClass-D for ORI, BON, and BLF using a prediction threshold of 0.5, and 2) comparing the systems' performance to that of the dermatologists based on ROC curves.
Performance evaluated on the MClass-D data set for the three models using a prediction threshold of 0.5: Figure 7 illustrates the performances of ORI, BON and BLF evaluated on the MClass-D dataset using a prediction threshold of 0.5. In general, BLF gives best performance in all three measures of AUC, SEN and SPE with values of 94.4%, 85.0% and 95.0% respectively, higher than ORI by 2.1, 5.0 and 5.0 percentage points respectively, and even higher than BON (the worst model) by 2.8, 20 and 1.2 percentage points respectively. Considering the sensitivity and specificity gap, BLF and ORI provide the smallest variation with 10%, while BON produces the worst with 28.8%. However, BLF performs best in terms of sensitivity (85%) and specificity (95%) 5 points higher than ORI (80% and 90%). These scores provide new state of the art on the MClass-D dataset. Comparison of the systems' performance to that of dermatologists based on ROC curves: Figure 8 shows the ROC curves of the three best models (in blue, green and purple, respectively for ORI, BON and BLF) and the performance of the 157 dermatologists (with red dots). In full details, the AUC of the three models BLF, BON and ORI respectively stands at 94.4%, 91.6% and 92.3%, much higher than that of dermatologists (67.1%). BLF is the best of all three models and has much better performance, being the first system to ever report results better than all of the 157 dermatologists involved in the system, with its AUC of 94.4%. The results of ORI and BON (92.3% and 91.6%) are better than those of 155 of the 157 dermatologists. All 3 models beat the state of the art, which only outperforms 136 of the dermatologists [11]. More detailed analysis of the prediction thresholds is provided in Table 5. In Table 5, SOTA stands the performance of Titus J. Brinker et al [11]. As shown in Table 2, the AUC of the dermatologists and of SOTA [11] were given for mean sensitivity and specificity of 74.1% and 60.0%. To ease comparison, Table 5 reminds these scores and provides the corresponding performance of all 3 models. At sensitivity 74.1%, BLF has a specificity of 97.5%, above the average of dermatologists by 37.3 percentage points, and above SOTA by 11 percentage points. With specificity at 60.0%, the sensitivity of BLF reaches 100%, while ORI and BON both have sensitivity of 95%. All three have very high sensitivity compared to the mean of dermatologists (74.1%) and SOTA (87.5%). Among the group of dermatologists, "chief Physicians" have the highest mean specificity with 69.2% at a mean sensitivity of 73.3%.
With the same specificity of 69.2%, BLF reaches a sensitivity of 90%, performing 16.7 percentage points higher than this subgroup of the most experienced dermatologists, and 5.5 percentage points higher than SOTA. However, at this level of specificity, ORI and BON actually perform event better in terms of sensitivity as they reach 95%.
In addition to raising the AUC, an objective of this study is to manage to balance sensitivity and specificity in order to manage data imbalance. In this respect, BLF proves most effective with sensitivity and specificity of 90% and 93.8%, surpassing the ORI (85.0% and 83.8%) and BON (85.0% and 82.5%). It can be seen that, at this balance level, BLF is better than ORI and BON. Both the sensitivity level of 90% and the specificity level of 93.8%, even more so their combination, are very high and applicable in actual practice.

Discussion
Our proposed binary melanoma classification system based on CNN with changes in batch logic, loss function and fully connected layers was trained with the most recent and biggest public datasets and it is the first one to outperform all of 157 dermatologists with junior and senior qualification from different hospitals. The previous bestperforming publication using the same MClass-D dataset did better than 136 out of 157 dermatologists. Besides this head-to-head comparison, on average also, our approach largely outperformed the existing state of the art as well as the group of dermatologists. First, the AUC of our proposed solution reached 94.4%, 27.3 percentage points better than the average of all the dermatologists and 23.3 points better than that of private practice and chief physicians. This study therefore brings new the state-ofthe-art AUC for binary melanoma classification on the MClass-D dataset. Second, with a prediction threshold of 0.5, our AI model achieves relatively balanced sensitivity and specificity at 85.0% and 95.0% respectively, much higher than the average of the physicians which lies at 74.1% and 60.0% respectively. Last, the output is a real number in the range of [0, 1]. Adjusting the threshold provides a simple and efficient way to put a stronger emphasis on either sensitivity or specificity. This is done as post-processing step, requiring milliseconds of processing time on a simple device. BLF remains the best model with threshold of 0.40858, with respective sensitivity and specificity of 90.0% and 93.8%. This is the best and most balanced indicator.
BLF is the best-performing model which used our optimized architecture of Deep CNN and both of our proposed changes to the batch logic and to the loss function. Besides, two other models were introduced, allowing to measure the impact of our 3 additions to the state of art. First, ORI, a model that uses only the optimized Deep CNN architecture but neither the loss function nor the batch logic. Second, BON, using the Deep CNN architecture and the batch logic, but not the loss function. Both of these models were also very effective, showing the key role of our optimized architecture of Deep CNN. They outperformed 155 dermatologists and respectively achieved AUC 2.1 and 2.8 points lower than that of BLF. In the evaluation on the Test-10 dataset, the AUC of ORI even exceeds that of BLF. This shows that the system architecture proposed using DenseNet169 in combination with customized fully connected layers is very effective for melanoma binary classification; it solves the underfitting and avoids overfitting problem.
The best-performance of BLF shows that an effective method to balance the two indicators of sensitivity and specificity in binary melanoma classification can be implemented by changing the loss function as detailed in Formula (4), fine-turning the parameters a, b and c so as to fit the datasets relevant to the adequate medical image classification tasks. We indeed believe that this result can be generalized to other applications. In the case of binary melanoma classification, the parameter values a = 1, b = 2 and c = 2 produce the best results. In addition, custom logic batch also affects the training process as it both helps to balance the indicators of sensitivity and specificity and to provide fast convergence.
Binary melanoma classification requires fully connected layers deep enough to avoid underfitting, and benefits from batch normalization and dropout to avoid overfitting. The combination of many fully connected hidden layers with batch logic and customized function loss gives excellent results, and exceeds the humans' accuracy and the previous state of the art for this task.

Conclusions
In this study, we have proposed a new approach to binary melanoma classification by reforming fully connected layers and customizing batch logic and loss function. The major results of this research are listed below: 1) The proposed system achieved state-of-the-art AUC of 94.4% at higher sensitivity of 85.0% and specificity of 95.0% and outperformed all of the 157 dermatologists on the MClass-D dataset [8]. The balanced performance of sensitivity and specificity at 90.0% and 93.8% when using a prediction threshold of 0.40858 is the state-of-the-art performance evaluated on the MClass-D dataset. 2) At sensitivity 74.1%, the proposed method has a specificity of 97.5%, exceeding the average of dermatologists by 37.3% and current SOTA by 11% [11]. Moreover, with specificity 60.0%, the proposed method reached 100% sensitivity, 25.9 higher than the mean of dermatologists and 12.5 higher than the current SOTA. 3) Using DenseNet combined with reforming full connected layers solved underfitting and avoided overfitting problems.
4) Optimizing loss function calculated by separate loss of the majority and minority class, and customizing batch logic improved the balance of sensitivity and specificity and obtained the best performances in terms of AUC, sensitivity and specificity.
The study is not only significant in terms of AUC, sensitivity and specificity for binary melanoma classification. The research results also show that customizing batch logic, loss function and reforming fully connected layers are very important features for a high-performing model to classify medical images. We intend to develop more intensive studies on loss function and fully connected layers, applied not only to melanoma classification but also to other medical image classification tasks.