{'fraction_non_alphanumeric': 0.03111111111111111, 'fraction_numerical': 0.016931216931216932, 'mean_word_length': 4.863986100769422, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Automated detection of pitting and stress corrosion cracks in used nuclear fuel dry storage canisters using residual neural networks

Nondestructive evaluation methods play an important role in ensuring component integrity and safety in many industries. Operator fatigue can play a critical role in the reliability of such methods. This is important for inspecting high value assets or assets with a high consequence of failure, such as aerospace and nuclear components. Recent advances in convolution neural networks can support and automate these inspection efforts. This paper proposes using residual neural networks (ResNets) for real-time detection of pitting and stress corrosion cracking, with a focus on dry storage canisters housing used nuclear fuel. The proposed approach crops nuclear canister images into smaller tiles, trains a ResNet on these tiles, and classifies images as corroded or intact using the per-image count of tiles predicted as corroded by the ResNet. The results demonstrate that such a deep learning approach allows to detect the locus of corrosion cracks via smaller tiles, and at the same time to infer with high accuracy whether an image comes from a corroded canister. Thereby, the proposed approach holds promise to automate and speed up nuclear fuel canister inspections, to minimize inspection costs, and to partially replace human-conducted onsite inspections, thus reducing radiation doses to personnel.


Introduction
Dry storage systems housing used nuclear fuel from commercial nuclear power reactors ( Figure 1) will be used for longer periods than initially anticipated and there is a concern that some of these systems may become vulnerable to physical degradation, e.g., pitting and chloride-induced stress corrosion cracking (SCC). The potential for SCC at the heat af-fected zones of welded stainless-steel interim storage canisters has been identified by several agencies [1,2,3,4,5,6,7,8]. Although this concern has motivated the development of delivery systems and inspection of these canisters on a regular basis [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25], the large number of stored canisters that need to be visually inspected (currently approximately 3, 200 canisters have been installed in the U.S. and projections show that eventually over 10, 000 canisters will be required by 2050 [26]), high radiation levels, limited access (through small size vents) and space constraints (overpack-canister gap less than 5-10 centimeters) make in-situ visual inspections challenging. This necessitates the development of remotely operated systems for real-time detection of flaws including type, location, size, and density, to support future remediation activities, reduce operator errors, and optimize repair quality. As a result, an integrated deep learning framework that would allow scanning a larger area and with high detection capabilities would be a big step forward to reduce dose-rate to operators, minimize inspection costs, and ensure long-term safety of used nuclear fuel canisters.
Existing methods for detecting structural defects in images include Frangi filters and Hessian affine region detectors [27], total variation denoising [28,29], combinations of image processing techniques for feature extraction [30,31] with machine learning algorithms for classification [32,33], and more lately deep convolutional neural networks (CNNs) [34,35]. So far, no deep learning approaches have been used to address inspection of used nuclear fuel canisters. Radiation levels typically create noise, flacking, snow-ing and other artifacts in cameras. Such artifacts pose a challenge for conventional feature detection techniques, e.g., shallow neural networks, that rely on clean surfaces, isolated defects, and radiation-free viewing with optimal lighting conditions. So, the process of inspecting used nuclear fuel canisters is not automated algorithmically and it induces a challenging image classification problem. Deep learning is a plausible approach for automating such a challenging classification problem. In fact, [35] have applied CNNs for detecting concrete cracks in radiation-free civil infrastructures. Along the lines of [35], this paper crops images to smaller tiles to train the underlying CNN.
The proposed approach is differentiated from [35] in three ways. First, ResNets are used for detecting cracks instead of deploying a custom CNN architecture. Training three different ResNets (ResNets-18, ResNet-34 and ResNet-50) demonstrates that the proposed approach tackles the crack detection problem without being sensitive to the choice of ResNet architecture. Secondly, the present paper introduces a classification rule for identifying images as corroded or intact using ResNets trained on tiles cropped out of these images. This way, the locations of potential cracks are linked to tiles predicted as corroded, Figure 2: Example image with visible corrosion, stress corrosion cracks, and artifacts (scratches, shadows, etc.). Note: this is a laboratory corroded stainless steel sample and not an image from an actual used fuel canister; to the best of our knowledge no canisters have been detected with defects).
while at the same time the problem of classifying the original images is solved with high prediction accuracy without being sensitive to tile-specific prediction errors. Finally, the proposed approach is focused on the specific application of detecting pitting and stress corrosion cracks in real data imagery similar to used nuclear fuel canisters.

Data
The research presented in this paper is based on data provided by the Electric Power Research Institute (EPRI). The data consist of 166 images taken from flaw mockup specimens produced by EPRI. These images, which capture stainless-steel canister surfaces, were taken with a 16-megapixel camera from a variety of locations, angles, and lighting conditions to induce high variability in the dataset. Each image has a 4, 928 × 3, 264-pixel resolution. Figure 2 shows an example corroded image. Interest is in detecting pitting or stress corrosion cracks in the images.
Use of deep learning for detecting canister cracks requires training sets of sample size larger than the 166 available images. For this reason, the 166 images were cropped into smaller tiled images of 256 × 256pixel resolution using a custom-made algorithm for Tiles with pitting or stress corrosion cracks (3a-3b) are labeled as corroded, while tiles without defects (3c-3d) or with scratches (3e-3f) or shadows (3g-3h) are labeled as intact.
this task, producing a total of 37, 719 tiled images (referred to as tiles thereafter). Each tile was labeled either as corroded or as intact, depending on whether it contains any cracks or not. Thus, the ground truth for image annotation takes into account only cracks identifiable via the naked human eye. The generated dataset of tiles includes a broad range of image variations (see Figure 3), which are necessary for CNN training.
The 256 × 256-pixel resolution for tiles was chosen empirically. Such a small resolution facilitates the training process of CNNs with tiled images coming from original images of varied resolution, therefore making the approach camera-independent and more generally applicable. On the other hand, tiles of smaller resolution may lead CNNs to mistake elongated features, such as scratches, for cracks. In addition, smaller tiles render their labeling less obvious to the human eye.
4, 957 out of the 37, 719 tiles were manually labeled as corroded, while the remaining 32, 762 were labeled as intact. After manually labeling each tile either as corroded or as intact, the dataset of tiles was split randomly into a training, validation, and test set, containing 22  sets independent, tiles from a particular image were included in only one set. Table 1a shows the number of corroded and intact original images while Table  1b shows the number of corroded and intact tiles in each of the three sets. An original image is corroded if it contains at least one tile manually labeled as corroded, otherwise it is intact. It is noted that the 60% − 20% − 20% split between training, validation, and test set is preserved at the level of original images too (99, 34 and 33 original images in the respective sets).

Brief overview of CNNs
A feed-forward CNN consists of multiple layers of units, starting with an input layer, followed by combinations of convolution, pooling, activation and fully connected layers, and ending with an output layer (see Figure 4). Convolution layers perform convolution operations to extract high level features from the input image. Convolution operations preserve the spatial relationship between pixels by learning image features using small squares of input data. Each convolution layer is typically followed by a unit-wise activation function, such as the rectified linear unit (ReLU). Pooling layers reduce the dimensionality of each extracted feature, but they retain the most important information. In a fully connected layer, each neuron is connected to every neuron in the previous layer. Fully connected layers learn non-linear combinations of the features identified by the convolutional layers. A common final layer in CNNs is the softmax layer, which assigns probabilities to each class label.
For a more detailed introduction to CNNs, see for example [36].

Proposed algorithm
In this paper, ResNets are used, which are a specific type of CNN architecture [37]. Each of the 7, 719 tile images in the generated dataset is passed to the ResNet input layer as a 256 × 256 × 3 tensor, whose three dimensions correspond to tile height, width and RGB channel (red, green and blue), respectively. The prediction made at the ResNet output layer indicates whether the input tile is considered to be corroded or intact. Raw images are cropped into tiles, which are in turn manually labeled and split into training, validation and test set. After training the ResNet on the training set, predictions are made for tiles coming from the test set. The heatmap of Figure 5 visualizes the tile predictions made by the ResNet, with redcolored and green-colored tiles predicted as corroded and intact, respectively.
The posed research question is to classify images as corroded or intact, whereas a ResNet provides predictions for fragments of the image. To solve the original problem, a rule for image classification can be defined by utilizing tile predictions. One classification rule is to predict the original image as corroded if it contains at least one tile predicted as corroded. However, such a classification rule is prone to produce false alarms due to being sensitive to tile prediction errors. Misclassifying a single tile as corroded is sufficient to produce a false alarm for the original image.
To make crack detection for an image less susceptible to its constituent misclassified tiles, the classi- assuming that 1 and 0 correspond to presence and absence of cracks, and c is a hyper-parameter.
The hyperparameter c sets a threshold on the count of tiles predicted as corroded, above which the image consisting of these corroded tiles is predicted to be corroded itself. This classification rule has been instigated by training ResNets on tiles. It has been observed empirically that images with cracks tend to contain higher number of corroded tiles than intact images with fewer (or no) tiles misclassified as corroded.
A value for the hyper-parameter c can be chosen by optimizing a performance metric, such as prediction accuracy per image, on the validation set. More specifically, if M ( ŷ (i) (c) : i ) is a performance metric, viewed as a function of image predictions ŷ (i) (c) : i dependent on c and made over the validation set, then c can be estimated byĉ = sup c M ( ŷ (i) (c) : i ).
To summarize, the proposed algorithm for detecting whether an image contains pitting or stress corrosion cracks comprises the following steps: 1. Run a ResNet on the validation set of tiles to tune hyperparameters relevant to training, such as the learning rate for stochastic gradient descent. 2. Train the ResNet on the training set of tiles. 3. Tune the value of hyperparameter c using the validation set of tiles. To do so, predict the labels of tiles in the validation set via the trained ResNet, use these tile predictions to compute the whole image predictions via equation (1) for various values of c, and select the value of c that maximizes a performance metric, such as the prediction accuracy or F1 score. 4. Predict the labels of images in the test set. To do so, predict the labels of tiles in the test set via the trained ResNet, and use these tile predictions to compute the whole image predictions via equation (1) for the value of c obtained at step 3.
As it can be seen from the described algorithm, a new image is cropped into tiles, its tiles are predicted as corroded or intact via the trained ResNet, and equation (1) is used for predicting whether the image is corroded or intact.

Tuning and implementation
This section provides details of the configuration of the training process, of hyperparameter tuning and of the implementation. To start with, data augmentation was performed by carrying out random rotations, Figure 5: Flowchart for classifying tiles using a ResNet. Solid lines represent training steps and dashed lines refer to test steps. Tiles from the test set, stitched back together to display the image of origin, are colored as red or green depending on whether they have been predicted as corroded or as intact by the ResNet. flips, color shifting, zooming and symmetric warping of the tiled images.
An 18-layer, a 34-layer, and a 50-layer ResNet were employed in order to validate the discriminative power of the proposed approach to crack detection regardless of the deployed ResNet architecture (see [37] for the definitions of ResNet-18, ResNet-34 and ResNet-50). Each ResNet was trained by running the one-cycle policy [38] for 20 epochs.
The learning rate was set to take values in [10 −6 , 10 −1 ] for training with the one-cycle policy. This interval of [10 −6 , 10 −1 ] was selected by running a mock training session with the learning rate finder of [39] prior to training, by plotting the curve of loss against the range of learning rates used in the mock run, and by choosing a part of the curve with downwards slope before the minimum loss. Figure 6a displays an example of a loss curve from a mock run of  the learning rate finder on ResNet-34.
The batch size was fixed to 128 for each of the three ResNets after mock training with batch sizes of 32, 64 and 128. For the empirical tuning of batch size, the image prediction accuracy of the proposed algorithm was maximized via mock training.
The threshold c of equation (1) for image classification was selected by maximizing the F1 score of image predictions on the validation set (see Section 3 for more details about the F1 score). Figure 6b  Simulations were run on an NVIDIA Tesla V100 GPU of a DGX-1 server at the CADES Cloud at Oak Ridge National Laboratory (ORNL). The training runtime for 20 epochs using the one-cycle policy varied from 15 minutes for ResNet-18 to 45 minutes for ResNet-50.

Results
A tiled image or an original image is characterized as positive if it is corroded (labeled as 1), and as negative if it is intact (labeled as 0). To summarize the results, Tables 2 and 3 provide the absolute frequency of true positives (TP; corroded correctly predicted as corroded), false positives (FP; intact wrongly predicted as intact), true negatives (TN; intact correctly predicted as intact) and false negatives (FN; corroded wrongly predicted as intact) for tiled image predictions made by the three ResNets and for whole image predictions based on equation (1), respectively. Tables 2 and 3 also include five additional performance  metrics for assessing the predictive capacity for tiled and for whole images, namely the true positive rate (TPR), false negative rate (FNR), positive predictive value (PPV), F1 score and prediction accuracy (ACC). The definition of thse metrics are available in the Appendix. Both ACC and F1 are plausible candidates for tuning the threshold c in equation (1) for the available data, since the classes of intact and corroded images are balanced with a sample size ratio of 82/84 ≈ 1 (see Table 1-a). The F1 score would be preferred over ACC for tuning hyperparameter c in equation (1) in the case of imbalanced classes of intact and of corroded images. As seen in 2, the respective accuracies ACC of tile predictions for ResNet-18, ResNet-34 and ResNet-50 are 95.95%, 95.34% and 96.97%, while the respective F1 scores of tile predictions are 78.99%, 78.34% and 96.97%. The accuracies are high regardless of ResNet architecture, while the F1 scores are lower, with the exception of ResNet50, which attains a high F1 score. The lower F1 scores in comparison to accuracies are related to the class imbalance between corroded and intact tiles. The probabilities of corrosion crack detection (TPR) correspond to 73.67%, 81.43% and 94.11%, while the probabilities of false alarms (FPR) correspond to 1.48%, 3.05% and 0%  for ResNet-18, ResNet-34 and ResNet-50.
Overall, ResNet-50 provides relatively high predictive accuracy for tiled images. The reason for providing the tile-level predictive accuracy of all three ResNet architectures becomes clear by consulting Table 3, which provides the predictive accuracy for the original images. As it can be seen, the image classification rule based on equation (1) is not sensitive to the choice of ResNet architecture. ResNet-18 and ResNet-34, which are characterized by lower tiled image-level predictive accuracy than ResNet-50, yield high whole image-level predictive accuracy. More specifically and at whole image-level, ResNet-18, ResNet-34 and ResNet-50 combined with the image classifier of equation (1)  Thereby, Table 3 provides empirical evidence that the image classifier of equation (1) is highly performing, as it is not sensitive to tile misclassification errors or to the choice of ResNet architecture for tile classi-fication. Thus, the proposed algorithm of Section 2.3 provides empirical evidence that it can address the research question of automatically detecting stress corrosion cracks in used fuel nuclear canisters. It is noted that based on the existing false alarm rates the proposed system would be best suited as an operator decision support system intended to augment existing human inspector capabilities, e.g., the operator would get a notification to evaluated whenever a corroded image is detected instead of having to evaluate all images one-by-one. A completely standalone and operator-independent detection system would require a very low rate of false alarms to avoid unnecessary time-consuming and costly re-inspections. A significantly lower false alarm rate could be achieved using a custom CNN trained on a much larger set of images than what was currently available.
A graphical user interface (GUI) was developed using PyQt to automate the process and analyze images. The GUI provides a colormap that highlights any identified corroded and intact areas within the image and provides additional information to the operator including percentage of image that is corroded. Figure 7 demonstrates GUI usage via an image classified as corroded. It is noted that this image was particularly challenging with a lot of artifacts (shadows, scratches, etc.). However, the proposed deep learning approach captured correctly all the corroded areas (highlighted with yellow color).

Discussion and conclusions
This paper proposes an image classification algorithm based on deep learning for detecting stress corrosion cracks in used nuclear fuel canisters. Despite the small sample size of 166 images, the algorithm attains high predictive accuracy and therefore encourages the future deployment as an operator decision support system intended to augment existing human inspector capabilities for detecting stress corrosion cracks in used nuclear fuel canisters. The proposed algorithm trains a ResNet on a training set of tiles, tunes the value of hyperparameter c using the validation set of tiles, and then uses tile predictions to compute whole image predictions. The proposed image classifier is highly performing and not sensitive to tile misclassification errors or to the choice of ResNet architecture for tile classification. Thus, the proposed algorithm holds promise for addressing the research question of automatically detecting stress corrosion cracks in used fuel nuclear canisters.
Future work includes a more exhaustive testing by acquiring a larger dataset of images from a diverse collection of corroded and intact samples and by allowing for class imbalances at whole image-level. Variations of the classification rule of equation (1) can be considered, for example by introducing sliding window techniques across tiles to take into account spatial information in tile predictions along with tile prediction counts. Another future direction would be to attempt pixel-wise labelling instead of cropping into 256 × 256 tiles, followed by a pixel-centric classification rule instead of the tile-centric classification rule of equation (1). Such an approach would require of an upfront investment to annotate images pixel-wise. A preliminary attempt to assess the proposed approach using video input instead of image input has been made, which takes about 15 seconds for ResNet training per input file. To enable realtime usage with video input, future work will use optical flows. Video files are not efficient for solving the corrosion classification problem because consecutive frames contain redundant information as the camera pans. Optical flows eliminate this issue and allow for the entire canister to be visualized by a single image. Thus, the proposed algorithm will be evaluated on images derived from optical flows.
Appendix A. Performance metrics TPR is defined as T P R = T P/(T P + F N ), it provides the probability of detection of corroded images, and it is therefore crucial for assessing corrosion detection in nuclear canisters. FPR is defined as F P R = F P/(F P + T N ), and gives the probability of false alarm. PPV is defined as P P V = T P/(T P + F P ), Figure 7: Visualization of an image identified as corroded using the GUI. The corroded areas are highlighted using yellow color and intact areas using green color.
and is particularly useful in the context of tile predictions, since the classes of intact and of corroded tiles are imbalanced with a sample size ratio of 37, 762/4, 957 ≈ 7/1 (see Table 1b). The F1 score is defined as F 1 = 2 × P P V × T P R/(P P V + T P R), and it is the harmonic mean of FPR and PPV. The overall prediction accuracy is given by ACC = (T P + T N )/(T P + T N + F P + F N ).

Acknowledgements
This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-publicaccess-plan). This work was funded by the AI Initiative at the Oak Ridge National Laboratory.