{'fraction_non_alphanumeric': 0.03224832417416511, 'fraction_numerical': 0.014554018962497736, 'mean_word_length': 4.792235047219307, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
"Prompt-Gamma Neutron Activation Analysis (PGNAA)"Metal Spectral Classification using Deep Learning Method

There is a pressing market demand to minimize the test time of Prompt Gamma Neutron Activation Analysis (PGNAA) spectra measurement machine, so that it could function as an instant material analyzer, e.g. to classify waste samples instantaneously and determine the best recycling method based on the detected compositions of the testing sample. This article introduces a new development of the deep learning classification and contrive to reduce the test time for PGNAA machine. We propose both Random Sampling Methods and Class Activation Map (CAM) to generate"downsized"samples and train the CNN model continuously. Random Sampling Methods (RSM) aims to reduce the measuring time within a sample, and Class Activation Map (CAM) is for filtering out the less important energy range of the downsized samples. We shorten the overall PGNAA measuring time down to 2.5 seconds while ensuring the accuracy is around 96.88 % for our dataset with 12 different species of substances. Compared with classifying different species of materials, it requires more test time (sample count rate) for substances having the same elements to archive good accuracy. For example, the classification of copper alloys requires nearly 24 seconds test time to reach 98 % accuracy.


I. INTRODUCTION
T HE composition of materials can be analyzed in a destructive way using traditional chemical process; nevertheless, we use an innovative material analysis The MetalClass project is funded by the German Federal Ministry of Education and Research. The copper alloys were provided by Wieland-Werke AG. We would also like to express our thanks to the pioneer researchers in deep image classification and other related fields, and to all members who have made a positive contribution to the MetalClass project. system, based on PGNAA technology and machine learning classification algorithms. The composition of waste material could now be classified in a non-destructive way and with advanced speed.
Neutron activation analysis (NAA) utilizes neutron-induced reactions to examine the spectra of the emissions of a radioactive sample, which could determine the element concentrations or material compositions of the sample. The sample is firstly bombarded by neutrons, causing the elements to form radioactive isotopes. The new forming activation product can be characterized through their radioactive emissions and the energy spectrum is detected by a sensor. and radioactive decay chain; therefore, metal compositions could be found. This project utilizes PGNAA [1], which is a subcategory of NAA. The additional prefix "PG" means prompt gamma-ray and hinted at state-of-the-art sensors which have a good resolution to detect the emission.
There is a strong marketing demand to apply PGNAA machines to production applications such as recycle factories or ore refineries [1]- [3]. If the necessary measurement time for PGNAA is a few hours, such measurements are only useful in a laboratory setting. The measuring time for production must be "heavily reduced". However, cutting the measuring time of PGNAA means the output spectrum is incomplete and noisy [6].
To deal with this incompleteness and noise, we devise Random Sampling Methods (RSMs) for generating the "downsizing" training samples, by assuming the spectral data as probability distributions. The result spectrum keep the intrinsic down-sized nature while the measurement time of the target sample is drastically cut. arXiv:2208.13909v1 [cs.
LG] 29 Aug 2022 An additional trick to downsize the samples -visualizing and indicating the importance of each energy level using Class Activation Map (CAM). We can then discard the less important energy range in order to reduce the necessary number of channels in detectors or to allow a finer resolution in the remaining energy ranged. We use these downsized samples to train Convolutional Neural Network (CNN) models. Since downsizing too much causes negative effects, we closely monitor loss, accuracy, training time, and prediction time throughout the training process. Visualization such as Confusion Matrix was done to ensure the prediction is accurate even the sampling time is minimized. Though other strategies may achieve better result in speed and accuracy [4], CNN is still an attractive tool for general classify purpose due to its interpretability of channel importance.

II. DATASET
We use totally three datasets for our experiments in Section IV. At the beginning of calibration test, the first dataset, with 10 species of substances (e.g. scrap metal powder, cement and stucco) and two species of soil, was used for Experiment I. However, we also collected two data sets of samples in the "same species". The second dataset is only with aluminium (for Experiment II and IIIa) and the third is with copper alloys from Wieland Electric GmbH (for Experiment IIIb). Each species creates a distinctive PGNAA spectrum. The kinetic energies of electron emission spread over a wide range of energy levels. An individual CSV file is dedicated to each species, which records "Energy (keV)" vs "Counts (intensities)". The following characteristics can be learned from a PGNAA spectrum: 1) Each constituent material of a sample could be identified by the characteristic peaks of PGNAA spectrum output. 2) Classification with sufficient measure counts is more accurate than with fewer counts; however, the resultant spectrum contains more information than necessary need then the measurement time is too long. If the measurement time is too short, the accuracy of the peaks will be poor, and noise will appear in the spectrum.
The minimum value of sample count rate to provide sufficient accuracy will be further discussed in Section IV-C.
3) The plottings of the substances having similar compounds are also close to each other because of the chemical similarity. These can be a challenge in this project.

III. METHODS
In order to construct a well-defined and modularized system architecture, we started our workflow with crucial procedures such as designing Random Sampling Methods, constructing models architectures, training and optimizing the training set-up. During this phase, we also have to foresee some of the future needs such as maintenance, revision, or expansion in the future.

A. Random Sample Methods (RSMs)
Random Sample Methods (RSMs) is a novel idea that could fulfil our main goal of reducing the measurement time [4], which is corresponded to "counts". The reason to conceive creating downsized samples with random methods is that their nature resembles the original PGNAA spectrum. Their similar nature help to identify the constituent substances and expedite the classification process. A complete data source can be collected all at once. The resulted sample spectrum is the skimmed version of its origin, and its measuring time is reduced tremendously. In other words, spectrum scanning of the target material takes originally few hours, and only a few seconds is allowed for our sample process. Hence, we generate down-sized samples for every training iteration. By applying RSM on the dataset, one detailed spectrum for each composition material is already enough train a model [7], [8], e.g. a CNN.
• For saving memory space, the data stored into a CSV file (in our data source) as two arrays: the above distinctive energy level list and its corresponding count list = [1, 10, 2, 0] are recorded.
For RSM, we randomly choose a smaller sample, corresponding to a shorter measuring time, for examples five samples (k = 5). The output of this sampling method is a downsized sample as a lengthy list of energy levels, and it might look like this: [2.6 keV, 1.3 keV, 2.6 keV, 3.9 keV, 2.6 keV]. Using the above distinctive energy level list = [1.3 keV, 2.6 keV, 3.9 keV, 5.2 keV] and counting the corresponding occurrence, the output of RSM is: • Result of RSM = corresponding count list = [1, 3, 1, 0] RMS can be repeated arbitrarily often, to lead arbitrarily many data points for training machine learning models.

B. CNN design for one-dimensional data
Convolutional Neural Network (CNN) has been widely used in the field of supervised learning [8] and especially with image classification. The filters (weights) are matrix values, which can be considered as feature extractor, are learned during the training phase of the model. A convolution between a filter and an image can induce effects such as sharpness, edge detection, and even the characteristic peaks of our spectrum.
After rounds of testing and comparisons, see Table II and [9], the Inception-ResNet-V2 model was usually significantly faster than their competitors and offer a better throughputaccuracy tradeoff. It also has relatively fewer model blocks, thus less work is needed for the modification in comparison with other complex architectures.
The following steps are taken to handle one-dimensional input data so that it can be compatible with our speedy RSM: • Replace/ add an untrained convolution layer with custom variants [10] at the initial point of the pretrained network ("Stem" in Inception model), so that the dimension of input data can be fed. • Set desired output classes. • Reduce all 2D layers to 1D including the kernel dimension.

C. Training and Fine Tuning
Besides optimizing the common training hyper-parameters, like batch size, optimizer, learning rate, etc., we also want to minimize our PGNAA data by (1) seeking the lowest count rate (i.e. live time) of PGNAA samples and (2) seeking energy channels that can be discarded for both CNN training and prediction.
1) The measurement count of a sample corresponds linearly to the measurement time of a sample; the fewer the count of measurements, the shorter the measurement time, and the more difficult it is to classify the sample. In short, the reduction of the sampling time is essential for this project, but excessive reduction of measurement time could lead to very noisy data, see Fig. 1. 2) Class Activation Map (CAM) can be used to illustrate the prediction decisions made by CNN, since Class Activation Map of a specific category indicates how CNN recognizes the distinguished image regions of that category [11].
As shown in Fig. 2, CAM is applied to squeeze the data further in order to expedite the classification process. The program's original setting for the CAM is two dimensions, the setting was changed to one dimension because our selected CNN Model requires one dimension; for instance, the input and the convolutions are one dimension. Base on the result graph, we discard Channel 8000 to Channel 16384 (5641.92 keV to 11552.48 keV) on the high energy band in the later setting for Experiment I.
The final decision is 0 to 103 on the lower energy band and C8000 to 16384 on the high energy band should be filtered out so that the accuracy and training time could be optimized.
Summarizing above, the final tuning of the best training for "10 species of substances and two species of soil" is shown as Table I.

IV. EXPERIMENTAL RESULTS
A. Experiment I: Train CNN with Dataset of 10 species of substances and two species of soil (live time ≈ 2.479 sec) 12 CSV files are used for training purposes which consists of 10 species of sample substances and two species of soil. These files fed into the RSM to be downsized and "smashed". Totally 19,000 samples are generated and fed into the CNN model for training which requires 150 epochs of iterations.
As shown in Table III, the trained CNN model makes good prediction and achieve good results; the accuracy is averagely about 96.88%. The prediction mostly matches with the true label. However, some misclassifications occurred (≈ 68.5%) that occurred between "Erdreich-11-15-30" and "Erdreich-HgS-inhomogen". This "same species" classification problem will be handled in the next experiment.
B. Experiment II: Dataset Aluminium with 20,000 as sample count rate (live time ≈ 1.11 sec) Using the same set-up as Experiment I, such as the same CNN model, same RSM, only the input dataset is using "Dataset Aluminium" and the discarded energy band of "0 -103" for this new training. The result is schown in Table IV Even when the training is only focused on the substances with the same species, the performance remains about 65 % and seems impossible to be raised. The problem can be caused by the setting of the sample count rate, in which the resulted sample data from RSM is too noisy and doesn't provide enough information to distinguish from similar species.
C. Experiment IIIa: Dataset Aluminium with increased sample count rate to 500,000 (live time ≈ 27.74 s) Totally five different sample count rates were tested. The curve with the lowest loss function and the best convergence is chosen. We started with the count rate (20,000) suggested by the last experiment. At the end, we choose 500,000 as our sample count rate according Fig. 3a, in which the loss function descents stably. The result is shown in Table V.
In addition, we visualized in Fig. 3b with CAM which energy ranges have more influence on classification and filtered out the irrelevant energy range to save training time.
D. Experiment IIIb: Dataset Copper with increased sample count rate to 500,000 (live time ≈ 24.44 s) For Dataset from Wieland's Copper, we repeated the same process as Experiment IIIa to choose the optimal sample count rate for training. The only difference is the CNN's high energy discarded range is set to the range of 14,000 to 16,384 (base on the suggestion from CAM) in order to improve the training speed. The result is shown in Table VI V. DISCUSSION All of our designed models, which equips with tested and chosen RSMs, can classify the PGNAA spectrum and find out the constituent materials speedily. The CNNs can classify the sampled PGNAA spectrum based on automated extracted aggregate features, which is different from common Machine Learning methods like Support Vector Machine (SVM) [14]. Other classical approaches such as Normalized Cross Correlation (NCC) [15] and Chi-squared matching, perform the classification by comparing the input data with templates iteratively. In order to achieve better accuracy, the number of these templates must be increased since they are proportionally related.

VI. CONCLUSION
The essence of this essay is to devise a suitable Random Sampling Method (RSM) to generate some downsized samples to train the CNN models instead of the cumbersome fully measured samples. The downsized samples are proven to be easier to handle and improving speed. Also, the downsized samples have a resemblance nature to the test spectrum reading from the PGNAA because both are heavily downsized data. These two reasons boost PGNAA's reading speed so fast that it becomes a high-speed scanner.
In the end, reduced Model (CNN Inception-ResNet-V2modified 1D) using RSM (Random weighted selection in counts) to extract our data from CSV, was determined as our best matching pair after several test runs. Besides tuning the model with different batch sizes, optimizers, and learning rates, the downsized training samples were also inspected with different sampling count rates so as to minimize our necessary sample live time for a prediction. In addition, to strengthen the downsizing process, some less important energy channels were filtered out according to the energy information provided by the Class Activation Map. For example, the highest energy range (Channel 8000 -16384) and the lowest energy range are discarded.

DATA AVAILABILITY
The MetalClas project is funded by the German Federal Ministry of Education and Research (BMBF) with grant number 01IS20082B. The copper alloys were provided by Wieland-Werke AG. The PGNAA data used to support the findings of this study are available in a public repository for AI Challenge Days 2021.

A. Comparing RSM and their best models
Four RSM were designed, but in section III-A only RSM-3 was introduced. The other three RSM are implemented using Python as follow: • RSM-1 is simply the lengthy measured energy value list. • RSM-2 applies a random binomial function to each row in the column 'Counts' of the CSV files. Nevertheless, the sum of measurement counts within a sample remains similar for each material. • RSM-4a or RSM-4b is a scatter-plot format or a histogram format converted from the sampling result from RSM-1.
The four RSM methods have to test together with DL models. In this section, CNN and other DL models were tested together with RSM.