{'fraction_non_alphanumeric': 0.028140411952422395, 'fraction_numerical': 0.01410646939367566, 'mean_word_length': 5.000217580504787, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A new lightweight deep neural network for surface scratch detection

This paper aims to develop a lightweight convolutional neural network, WearNet, to realise automatic scratch detection for components in contact sliding such as those in metal forming. To this end, a large surface scratch dataset obtained from cylinder-on-flat sliding tests was used to train the WearNet with appropriate training parameters such as learning rate, gradient algorithm and mini-batch size. A comprehensive investigation on the network response and decision mechanism was also conducted to show the capability of the developed WearNet. It was found that compared with the existing networks, WearNet can realise an excellent classification accuracy of 94.16% with a much smaller model size and faster detection speed. Besides, WearNet outperformed other state-of-the-art networks when a public image database was used for network evaluation. The application of WearNet in an embedded system further demonstrated such advantages in the detection of surface scratches in sheet metal forming processes.


Introduction
In industrial production, the detection of workpiece surface defects is essential to ensure product quality [1]. For example, in a metal forming process, surface scratch in contact sliding has been critical because it downgrades the surface quality of a workpiece and the service life of a tooling system. However, this has been traditionally a manual process, which is very inefficient, inaccurate and unreliable [2]. Recent advances in artificial intelligence provide a promising approach to tackling tough engineering problems, such as fault detection [3], nonlinear system control [4,5] and surface defect detection [6]. For instance, the image features of surface defects can be learned by machine learning techniques such as the support vector machine (SVM) and the artificial neural network (ANN). The SVM algorithm has been utilised to analyse and classify surface defects on steel surfaces [7] and cutting tools [8]. Similar applications of the ANN algorithm have been noted for defect recognition in cold rolling [9] and colour-filter production [10]. For example, the performance of different neural networks for surface crack detection in fracture experiments was tested and compared [11]. However, the separated feature extraction and classification operations have significantly restricted the detection efficiency.
The CNN-based deep learning technology has demonstrated its capability in image classification because it can automatically detect and extract high-level image features from the labelled image data [12]. Several CNN networks have been developed and applied to classify the image data, e.g. AlexNet [13], VGG-16 [14], GoogleNet [15] and Effi-cientNet [16]. The applications of CNN networks for the detection of surface defects [17,18], rolling bearing degradation [19] and roll marks on hot-rolled steel plates/strips [20] were also noted. For example, an image detection model based on R-CNN network was proposed to identify the wear location and wear mechanism in tribological tests [21].
Besides, different optimisation algorithms have been proposed to train the CNN networks appropriately. For instance, an Adam optimiser with power-exponential learning rate was proposed to control the iteration direction and step size in order to tackle the problems of local minima and overshoot in network training [22]. Although the CNN networks have been widely used for image classification with high accuracy in recent studies, their large model sizes and complicated structures limit the classification speed and bring about high latency.
Therefore, lightweight CNN networks, e.g. SqueezeNet [23], MobileNet [24] and ShuffleNet [25], have been developed to decrease the network parameter number and model size without sacrificing the classification accuracy. For example, a new fire module was utilised in the SqueezeNet to considerably reduce the computation consumption and communication cost [23]. Therefore, it can be feasibly built in the hardware with limited memory, e.g. mobile devices, to complete the real-time object detection in automatic visionbased systems [26]. However, a lightweight CNN network for detecting the surface scratch in contact sliding is yet unavailable.
Recently, the embedded system has been used to deploy CNN networks to complete real-time recognition and detection tasks, e.g. vehicle plate recognition [27], fire detection [28], handwriting recognition [29] and action recognition [30,31]. Normally, the embedded hardware has limited computation capacity and on-board memory; thus, lightweight CNN architectures are more feasible to be deployed in the embedded environment. For example, an anamorphic depth lightweight CNN, Anam-Net [32], was proposed to segment anomalies in COVID-19 chest CT images. Therefore, it is expected that deploying a lightweight CNN network for surface scratch detection in sheet metal forming will help to improve the level of automation and efficiency. This paper aims to develop a lightweight CNN structure, called WearNet, for surface scratch detection in contact sliding. A customised convolutional block will be developed to reduce the training parameter number and network layers as well as to simplify the network structure but retain classification accuracy. To train the WearNet, cylinder-on-flat sliding tests will be conducted to provide a large-scale surface scratch dataset. The network response and decision mechanism will be investigated to reveal the WearNet capability. The WearNet will then be compared with the existing advanced CNN structures to demonstrate its advantages in classification accuracy, model size and computation efficiency. In addition, the performance of the developed WearNet will be compared against other existing CNN networks based on a public image database, i.e. the NEU surface defect database [20]. Finally, a Linuxbased embedded system will be utilised as the deploying environment to further test the detection performance of WearNet.

Experimental setup for data collection
To develop a reliable CNN-based detection model, a large-scale surface scratch database is essential. To extend the database scale, cylinder-on-flat sliding tests (see Fig. 1) were conducted under a wide range of operation conditions listed in Table 1. The cylinder-on-flat sliding setup has been used to mimic the contact conditions encountered in a metal forming process [33,34]. To match industrial production conditions, two typical types of high-strength steel, DP980 and QP980, and DC53 tools with nitriding and vacuum heat treatment were selected as the pair of sliding contact. Both the oneway and cyclic sliding tests were carried out to mimic practical sliding types. The ranges of testing parameters  Table 1.

Image data processing
After each sliding test, the surface topography was measured by a digital microscope, OLYMPUS DSX 510. The measuring size of each image is 750 × 750 µm. Both of the surface images of DP980 and QP980 workpieces were divided into five categories (see Fig. 2): 1. The surface images prior to contact sliding were labelled as intact surface. 2. After certain cycles in a sliding test, if material transfer was identified on the workpiece surface without obvious scratches, the measured surface images were denoted as material transfer. 3. The images with the maximum depth of scratching (h max ) below 2 µm, 2 < h max < 4 µm and h max > 4 µm were called as minor, mild and severe scratch, respectively.
The surface images, Fig. 2c-e, were identified based on the h max . This is because h max plays an important role in determining the severity of scratching damage [35,36]. Overall, the database with a total of 10,500 surface images was identified by ten labels, as shown in Table 2. These images were randomly divided into training, validation and testing datasets with a ratio of 4:1:1 (7000:1750:1750). The image resolution was normalised to 227 × 227 prior to the training.
3 WearNet for surface scratch detection

Structure of WearNet
The existing advanced CNN networks were designed to classify the 1000 labels of ImageNet [37,38] with a database of over 14 million images. For the surface scratch identification in the current study, the image database was much smaller than ImageNet and fewer image labels were utilised. As such, a lightweight WearNet was developed based on a novel convolutional block to prevent overfitting, to effectively minimise the network parameters and to reduce the model size. The architecture and specifications of WearNet are outlined as follows (see Table 3 and Fig. 3): 1. Convolutional layer: this plays an important role in extracting the image features from the input image data, which is achieved by the convolution kernel. The con- volution kernel is a square filter, which can scan the input image and output feature maps. The kernel sizes for conv-1 and conv-2 are 3 × 3 and 1 × 1, respectively. Each convolutional layer is followed by a ReLU activation function.
2. Max-pooling layer: The role of the pooling layer is to reduce the feature map size by downsampling. There are two common methods to conduct pooling: average pooling and max pooling. The max-pooling is more suitable for image feature processing as it preserves the maximum output in a rectangular region. Therefore, the max-pooling strategy was selected in this paper. Besides, the network conducts max pooling with a stride of 2 to ensure late downsampling. 3. Convolutional block: This block takes advantage of separable convolution and squeeze-expand operations, as shown in Fig. 4. It starts with a squeeze convolution layer with 1 × 1 filters, which helps to restrict the total number of input channels, n 1 , fed into the following expand module. The expand module consists of batch normalisation, separable convolution and expand convolution using a 3 × 3 kernel. In the separable convolution, a channel-wise 3 × 3 spatial convolution is followed by a point-wise 1 × 1 convolution, which can bring about higher computation efficiency as fewer convolution operations will be conducted. In the concatenation layer, the channel number increases from n 1 to 4 × n 1 . As such, the network parameters and model size are decreased significantly.
4. Dropout layer: The dropout layer is to avoid the overfitting problem in the network training process [39]. The strategy is to deactivate some hidden layer nodes in the neural network and minimise their effects in the current training step. In this study, a dropout layer with a ratio of 0.5 was applied after max pool-3. 5. GAP layer: The global average pooling (GAP) layer is used to replace the fully connected layer in the traditional CNN networks. The GAP layer averages each feature map to enforce the correspondence between feature maps and image categories. There is no optimisation for the parameters, which further reduces the network parameters and minimises the overfitting problem.
In this study, the WearNet was developed by using the customised convolutional block. The network layers and parameters were effectively minimised to bring about a smaller mode size and higher classification speed. The comparison among the WearNet and other CNN networks was conducted on an embedded system to demonstrate the distinguished performance of the proposed WearNet for practical applications.

Training details
The WearNet was trained and evaluated by MATLAB on a PC with an Intel i5-10,600 (3.3 GHz and 16 GB RAM) and an NVIDIA RTX 3080 GPU (10 GB). Deep Learning Toolbox in MATLAB can provide a friendly framework for building network structures, setting training parameters and monitoring training processes. GPU computing was utilised in the network training to speed up the iteration. In general, there are three learning algorithms in the machine learning area, including supervised learning [40], unsupervised learning [41] and reinforcement learning [42]. In this paper, the supervised learning algorithm was adopted and the dataset consisting of labelled images listed in Table 2 was used for network training. The selection of training parameters (e.g. learning rate, gradient algorithm and mini-batch size) is discussed in the following section.

Evaluation protocol
The evaluations of deep neural networks are based on the following aspects:  where N a and N refer to the numbers of accurately classified images and total images, respectively. Classification accuracy is one of the most important evaluation metrics, as it indicates the overall classification performance of a CNN network. However, the classification accuracy alone can be misleading, if the numbers of surface images in individual classes are unequal. Therefore, the recognition rate and confusion matrix are used to check the network performance on each image class and to figure out how the CNN network is confused when making classification decisions. The training and classification time will play an important role when the computation efficiency of different networks is investigated. Besides, the model size should be taken into consideration when the CNN network is deployed in the embedded environment, as the on-board memory of an embedded device is usually limited.

Results and discussions
In training the WearNet, it is crucial to select appropriate training parameters. This should be done by considering the network structure, the surface image database and the computation resource available. In this section, the selection of the optimised training parameters for WearNet was explored. Then, the WearNet was investigated by focusing on the network response, layer activations and network decision mechanism. The comparison between the WearNet and other CNN networks was conducted by using the evaluation protocol in the last section.

Selection of training parameters
The effects of different training parameters, e.g. learning rate, gradient algorithm and mini-batch size, were investigated in this paper, as they were reported to have a considerable influence on the training results in the literature [43,44]. In the training experiments, it was found that the validation accuracy usually reached a stable stage after around 150 training epochs. Therefore, a series of network training experiments were conducted with the maximum epoch number of 200. The epoch refers to the entire image database being fully trained once. The mini-batch size (N b ) refers to the number of image data used for networking training in a single iteration. Generally, the network parameters are trained and updated by a greater number of times for a higher epoch number. With a given training database size D, the number N i of total iterations can be given by 1. Learning rate: The learning rate determines the converging speed of iteration. In the network training, it is essential to find an optimal value of learning rate to achieve a reasonable balance between the training speed and validation accuracy. Different learning rates, ranging from 0.001 to 0.00001, were tested and compared, as shown in Fig. 5. Besides, a piecewise learning rate from 0.001 to 0.0001 was also utilised in the training experiments. The descent algorithm and mini-batch size were set as stochastic gradient descent method (SGDM) and 16, respectively. According to the training results, it is concluded that a larger learning rate enables the model to learn faster but brings with it a risk of sub-optimal results. When the learning rate becomes smaller, the convergence speed becomes lower in the initial stage, and it takes a longer time to reach the stable stage. In particular, if the learning rate is too small (e.g. 0.00001), the final validation accuracy is relatively lower after 200 training epochs. Therefore, the piecewise learning rate from 0.001 to 0.0001, which combines the helpful characteristics of the larger and smaller learning rates, can bring about a fast convergence in the beginning and ensure a high validation accuracy in the final stage. Figure 5 also presents the average training time for single epoch, indicating that the influence of learning rate variations on training time is negligible. 2. Gradient algorithm: The gradient descent algorithm is used in the training of deep neural networks [45,46]. This section compares the training performance of two typical gradient algorithms, SGDM and Adam, and selects the appropriate algorithm by considering the convergence speed, computation efficiency and generalisation ability. Compared with the traditional gradient descent algorithms, SGDM computes the gradient of the loss function only by a small random subset, instead of the whole dataset, and performs a parameter update, which can help to improve the computation efficiency. The Adam algorithm utilises squared gradients to scale the learning rate and takes advantage of momentum by the moving average of gradient. Figure 6 presents the training process of the two algorithms with the batch size and learning rate fixed at 16 and piecewise, respectively.
Because of the random gradient computation, the SGDM usually has a lower convergence speed in the beginning and reaches its stable stage after a higher number of iterations compared with the Adam algorithm. However, the former consumes less training cost and leads to a higher validation accuracy than the latter. This is because more frequent updates are conducted for SDGM. Hence, there are more chances to jump out of a local minimal and search for better solutions. Hence, the SGDM algorithm was adopted in this study.
3. Mini-batch size: In training the CNN networks, the scale of image database is usually very large. The computational cost will be unaffordable if the whole database is swept in each iteration. Hence, a proper selection of a mini-batch size is important to reduce the training cost and refine classification performance [47]. By using the SGDM and piecewise learning, Fig. 7 demonstrates how the mini-batch size affects the training cost and validation accuracy. In general, the influence of batch size variations is more significant at the beginning of network training. small. It is also found that a larger batch size may lead to poorer generalisation ability. For example, as shown in Fig. 7, the validation accuracy drops gradually when the batch size increases from 16 to 64. With considering the balance between the training cost and validation accuracy listed in Fig. 7, the mini-batch size will be fixed at 16.

Examination of developed WearNet
With the above selected parameters, the WearNet for surface scratch detection was trained. The network responses, layer activations and decision mechanism of the WearNet are as follows: 1. Network responses: To examine the WearNet responses, a t-distributed stochastic neighbour embedding (t-SNE) function [48] was used in the study. Three hundred surface images with six different labels, as listed in Table 4, were used to investigate the responses of different layers, i.e. maxpool-1, conv-2 and softmax, in the WearNet. Figure 8 illustrates the t-SNE plots for three different layers where the six colours of these solid dots refer to the six image labels. For the maxpool-1 layer, the labels were not correctly grouped because only lowlevel features, e.g. colours and edges, were operated in such an early layer. The conv-2 layer can refine the cluster of these labels to some extent, but the accuracy was not satisfactory. For the softmax layer, the t-SNE plotting demonstrates that an appropriate classification of these different labels was achieved as the network went deeper, which validated the high accuracy of the developed WearNet.
2. Examination of layer activations: The layer activations play an important role in training the WearNet. To check which features the network has learned and whether the representative features have been correctly detected and preserved, it is necessary to visualise and examine the activation maps within different layers, i.e. conv-1, squeeze layer in conv-block-2 and conv-2. A testing image, QP980 surface with severe scratch, was fed into the trained WearNet.
There are 32 channels in the first convolution layer (conv-1). The 32 image features corresponding to the 32 channels are shown in Fig. 9a. Similarly, Fig. 9b, c present the feature  maps of the squeeze layer in the conv-block-2 and the last convolution layer (conv-2). The numbers of channels of the squeeze layer and conv-2 layer are 32 and 10, respectively. Figure 10 illustrates the image features with the largest activations in the three layers. Clearly, the discriminative features for characterising the severe scratch were extracted step by step as the neural network went deeper. For example, the deep and long ploughings were typical features for severe scratching images (Fig. 10).
3. Decision mechanism: To figure out how the WearNet makes a reliable classification decision, the gradientweighted class activation mapping, Grad-CAM [49], technique was employed. It utilises the gradient of the final classification scores associated with the convolutional features to determine the most influential part of a tested image for the classification. Figure 11 illustrates the Grad-CAM map for a test image from the class of QP980-severe scratch. The regime with the blue refers (a) conv-1 layer (b) squeeze layer in conv-block-2 (c) conv-2 layer to a low influence while the regime with the red denotes a high effect. In general, a larger gradient corresponds to the red zone on which the final score most relies. It should be noted that the red rectangle zone also refers to the area with a deep and long scratch, which has the greatest impact on classifying the test image as severe scratch.

Comparison with other CNN networks
A series of classification experiments were conducted to compare the performance of the WearNet with other state-of-the-art networks when the training conditions, i.e. image dataset (see Table 2), training platform and parameter settings, were identical.  Figure 12 presents the evolution of validation accuracy during the network training, in which WearNet has the highest value throughout the   whole training process. Furthermore, Table 6 presents the recognition rates of individual image labels when different CNN networks are tested. In general, the recognition rates of most image labels are over 85%. When it comes to the labels DP980-severe scratch and QP980mild scratch, the WearNet is still able to provide reliable classification results, while the recognition rates related to other networks drop significantly, especially AlexNet. Therefore, Fig. 13 shows the confusion matrices for AlexNet and WearNet, which can help to figure out how two CNN networks are confused when making classification decisions. It can be found that for the scratch images with the label of DP980-severe scratch, AlexNet is able to classify them with a recognition rate of only 65%, while around 7% and 26% are incorrectly classified as DP980-minor scratch and DP980-mild scratch, respectively. However, when the WearNet is employed, the classification error can be reduced significantly, as shown in Fig. 13b. Table 7 compares the complexity of WearNet and other CNN networks in terms of layer number, parameter quantity, model size and classification time. It is found that the model size is closely related to the quantity of network parameters, while the number of network layers has a considerable impact on its classification time. As shown in Table 7, the structure of WearNet is simpler than that of its convolutional counterparts, which brings about the smallest model size and fastest classification speed. However, the WearNet still has excellent classification performance, which should be attributed to its well-designed lightweight architecture. In conclusion, the WearNet proposed in this study has shown its advantages in computational efficiency and model size, as well as in its excellent classification performance.

Classification performance on a public dataset
To comprehensively demonstrate the effectiveness of WearNet, its performance is investigated based on the NEU surface defect dataset. This dataset collects six kinds of typical surface defects on hot-rolled steel strips (see Fig. 14), i.e. rolled-in scale, patches, crazing, pitted surface, inclusion and scratches, with a set of 300 labelled images for each type. The surface defect images in the database are greyscale and will be converted into RGB images before being used for network training. All the images were randomly divided into training, validation and testing datasets with a ratio of 4:1:1 (1200:300:300). A smaller training epoch (100) was adopted due to the smaller size of the image database, while other training parameters were identical to those in the last section. Table 8 presents the classification performance of four different CNN networks. All the four networks achieved high validation and test accuracy (over 98%), but the WearNet

Deployment of CNN networks
Currently, embedded systems are widely used in the industry due to their advantages in high efficiency, good affordability, continuous production and low energy consumption. In this study, a Linux-based embedded system, in which Raspberry Pi 4B works as the core hardware (see Fig. 15), was used to further demonstrate the application of CNN networks. In this section, 600 surface images selected from the testing dataset were used in the surface defection test. Table 9 compares the classification of surface scratch in the embedded system with four different CNN structures. The folder size refers to the total size of whole configuration files, which enables the embedded system to run detection programmes independently. In the embedded environment, the folder size and classification time of the WearNet were significantly lower than others, while its testing accuracy was still the highest. Hence, it is expected that WearNet will have promising prospects in industrial production.

Conclusions
In this study, a lightweight CNN structure, called WearNet, has been developed based on the well-designed convolutional block. The WearNet is designed for surface scratch detection in contact sliding, and the surface scratch images in the database are collected from cylinder-on-flat tribological tests. A detailed investigation on the parameter selection for network training and examinations on the network response and decision mechanism have been carried out. The performance comparison between the Wear-Net and other commonly used CNN networks has been conducted by using different databases. The main contributions of this paper are summarised as follows: 1. The developed lightweight WearNet has minimised network layers and parameters, with distinguished advantages in model size and classification speed, while guaranteeing high classification accuracy and recognition rate. 2. Training parameter variations have a significant influence on the network training process. The selected combination of training parameters manages to achieve a good balance between computation consumption and network performance. 3. The developed WearNet is able to extract and learn discriminative features for surface scratch classification step by step. The examination results demonstrate the excellent capability of WearNet to correctly classify different scratch images with appropriate labels. 4. The application of WearNet in an embedded system shows that WearNet has promising application prospects in production.