{'fraction_non_alphanumeric': 0.03603338448924675, 'fraction_numerical': 0.016839405470178904, 'mean_word_length': 4.8279833374173, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs

Deep Neural Networks (DNNs) are used in a wide variety of applications. However, as in any software application, DNN-based apps are afflicted with bugs. Previous work observed that DNN bug fix patterns are different from traditional bug fix patterns. Furthermore, those buggy models are non-trivial to diagnose and fix due to inexplicit errors with several options to fix them. To support developers in locating and fixing bugs, we propose DeepDiagnosis, a novel debugging approach that localizes the faults, reports error symptoms and suggests fixes for DNN programs. In the first phase, our technique monitors a training model, periodically checking for eight types of error conditions. Then, in case of problems, it reports messages containing sufficient information to perform actionable repairs to the model. In the evaluation, we thoroughly examine 444 models -53 real-world from GitHub and Stack Overflow, and 391 curated by AUTOTRAINER. DeepDiagnosis provides superior accuracy when compared to UMLUAT and DeepLocalize. Our technique is faster than AUTOTRAINER for fault localization. The results show that our approach can support additional types of models, while state-of-the-art was only able to handle classification ones. Our technique was able to report bugs that do not manifest as numerical errors during training. Also, it can provide actionable insights for fix whereas DeepLocalize can only report faults that lead to numerical errors during training. DeepDiagnosis manifests the best capabilities of fault detection, bug localization, and symptoms identification when compared to other approaches.


INTRODUCTION
Deep Neural Networks (DNNs) are becoming increasingly popular due to their successful applications in many areas, such as healthcare [23,30], transportation [38], and entertainment [17]. But, the intrinsic complexity of deep learning apps requires that developers build DNNs within their software systems to facilitate integration and development with other applications. The construction of such systems requires popular Deep Learning libraries [14,28].
Despite the increasing popularity and many successes for using Deep Learning libraries and frameworks, DNN applications still suffer from reliability issues [21,22,42]. These faults are harder to detect and debug when compared to traditional software systems, as the bugs are often obfuscated within the DNNs. Therefore, it is important and necessary to diagnose their faults, and provide actionable fixes. To that end, software engineering research has recently focused on improving the reliability of DNN-based software. For instance, there have been studies on characterizing DNN bugs [21,22,42], on testing frameworks for deep learning [37], on debugging deep learning using differential analysis [26], and fixing DNNs [11,41,43]. There are also frameworks and tools for inspecting and detecting unexpected behavior in DNNs. However, they require that specialists verify the visualization, which is only available upon completing the training phase [3-5, 27, 34].
Due to the complexity of using existing frameworks to debug and localize faults in deep learning software, recent SE research has introduced techniques for automatically localizing bugs [39,44]. DeepLocalize performs dynamic analysis during training to localize bugs by monitoring values produced at the intermediate nodes of the DNNs [39]. If there is a numerical error, then this approach traces that back to the faulty layer. DEBAR [44] is a static analysis tool that detects numerical errors in the DNNs. While both approaches have significantly advanced the state of the art in debugging DNNs, they do not detect bugs that manifest as trends of values (e.g. vanishing gradient, exploding gradient, accuracy not increasing) and do not offer possible fixes.

arXiv:2112.04036v1 [cs.SE] 7 Dec 2021
We propose DeepDiagnosis (DD), an approach for localizing faults, reporting error symptoms, diagnosing problems, and providing suggestions to fix structural bugs in DNNs. Our approach introduces three new symptoms of structural bugs and defines new rules to map fault location to its root cause in DNN programs. We implemented DD as a dynamic analysis tool and compared and contrasted it against state-of-the-art approaches. DD outperforms UMLAUT [33] and DeepLocalize [39] in terms of efficiency and AUTOTRAINER in terms of performance [43]. For example, assume the unchanged weight symptom, which occurs when the weights in the network the output are not changing for several iterations. In that case, DD would identify the root cause as that the learning rate is too low or that the optimizer is incorrect and recommend a fix.
In summary, this paper makes the following contributions: • We study different types of symptoms and propose a dynamic analysis for detecting errors and recommending fixes. • We introduced DeepDiagnosis (DD) the reference implementation of our approach. • We evaluated DD against SoTA. We found that DD is more efficient than UMLAUT [33] and DeepLocalize [39]. Also, DD has better performance than AUTOTRAINER [43]. • We provide a set of 444 models that practitioners can use to evaluate their fault localization approaches. • We make DD available, its source code, evaluation results, and the problem solutions for 444 buggy models at [6]. The rest of the paper is organized as follows: §2 describes the motivation of our approach. §3 describes our dynamic failure symptoms detection algorithm. §4 describes the evaluation of our approach compared with prior works. §5 discusses the threats to validity. §6 discusses related works, and §7 concludes and discusses future work.

A MOTIVATING EXAMPLE
In this section, we motivate our work by providing an example to illustrate the complexity of localizing faults and reporting their symptoms in DNN programs.

Listing 1: Bad Result for Simple Model [2]
Consider the code snippet in Listing 1 from Stack Overflow [2] with an example of a DNN. This model showed erratic behavior during training and returns bad results. At line 1, the developer constructed a sequential model and added a dense input layer at line 2 with the activation functions relu specified at line 3. Then the developer added a dropout layer at lines 4 and 7. Lines 5 and 8 are dense hidden layers with the activation functions relu and softmax specified at lines 6 and 9, respectively. The developer then compiled the model at line 10 and trained it using the fit() function at line 11. When compiling, the developer must specify additional properties, such as loss function and optimizer. In this example, the developer used as loss binary_crossentropy and optimizer RMSprop() at line 10. Finally, at line 11, the developer specifies the training data, batch_size, epoch, and validation_data.
The developer noticed that the DNN program was providing bad accuracy and could not diagnose the problem nor fix it (Stack Overflow post [2]) while following the Keras MNIST example [14].
The main issue with the code in Listing 1 is that it handles a binary classification problem, and therefore it should not use the activation function softmax in line 9. As the softmax works for multi-class classifications problems. Instead, it should use sigmoid, as it is the best suited for binary classification and will provide the best accuracy for the task. The current state-of-the-art for DNN fault localization is limited in terms of speed, accuracy, and efficiency. Table 1 summarizes the analysis results from three tools (DeepLocalize [39], UMLUAT [33], AUTOTRAINER [43]) and our approach DeepDiagnosis to diagnose the DNN model in Listing 1. To apply UMLUAT for the above example, we made semantic changes that were validated by the authors [33]. After 104.65 seconds, the training was terminated, with UMLAUT not reporting any problems. To apply DeepLocalize, we followed the instructions in the GitHub repository [7]. DeepLocalize prints the following message after 2.14 seconds: "Layer 7: Numerical Error in delta Weights. " This message indicates that there is a numerical error in the backpropagation stage during training. Indicating fault location, but it does not help developers to fix the problem. To apply AUTOTRAINER, we followed the instructions in the GitHub repository [8]. After performing the training phase, AUTOTRAINER did not solve the problem and took 495.83 seconds. Specifically, AUTOTRAINER detects a Dying ReLU symptom, but it does not provide the fault location -whether if it is in line 3 or 6. AUTOTRAINER tries to automatically fix the issue by trying different strategies (i.e., substituting activation functions, adding batch normalization layer, and substituting initializer), which, unfortunately, are unsuccessful.
Our approach DeepDiagnosis correctly reports the fault location and its symptoms after 35.03 seconds. Also, it provides a suggestion to perform a fix in the form of a message. Specifically, DeepDiagnosis reports that the bug is located in the backpropagation stage of layer 7 at line 8. Also, it prints out a numerical message: "Error in delta Weights", which indicates the type of the symptom. It also reports that the root cause is the activation function in layer 8 at line 9 (softmax). Finally, it answers the developer's question -there is indeed a problem with the activation function in the last layer and not in the training dataset.

APPROACH
In this section, we provide an overview of our approach for fault localization. We provide descriptions of failure symptoms and their root causes. Also, we describe the process of mapping symptoms to their root causes. Our approach monitors the key values during training, like weights and gradients. During training, it analyzes the recorded value to detect symptoms and determine whether a training problem exists. If a symptom is detected, our approach invokes a Decision Tree to diagnose/repair information based on a set of predetermined rules. Otherwise, the training will terminate with the trained model and reported the model is correct. Figure 1 shows an overview of our approach for fault localization, DeepDiagnosis, and for suggesting locations fix. DD starts by receiving as input the initial model architecture with a training dataset and passing our callback method as a parameter to the fit() method (Figure 1 left component). This callback allows capturing and recording the key values (i.e., weight, gradient, etc.) during feed-forward and backward propagation stages (Figure 1 middle component). Then DD applies a dynamic detector during training to report different symptoms at different stages based on error conditions (see Section 3.2 for more details). If DD detects a symptom, it further analyzes the recorded key values to determine the input model's probable location for the fix (Figure 1 right component). Finally, DD reports the symptom type, which layers and stage the symptom was detected, and suggests a location fix.

Failure symptoms and root causes
Our goal is to detect failure symptoms as soon as possible during development. So that if the model is incorrect, developers would not have to wait until the end of the training to find that model has low accuracy, thus wasting computational resources. To that end, we collected 8 types of failure symptoms and their root causes from previous work in the AI research community [19,20,29,35]. We provide more details of each of the symptoms and their root causes below.
3.2.1 Symptom #1 Dead Node. The Dead Node symptom takes place when most of a neural network is inactive. For example, assume that most of the neurons of a DNN are using the ReLU activation function, which returns zero when receiving any negative input. If the majority of the neurons receive negative values (e.g., due to a high learning rate), they would become inactive and incapable of discriminating the input. The DNN would end up with poor performance [43]. To identify this symptom, we compute the percentage of inactive neurons per layer. If the majority of the neural network is inactive, then we call it Dead Node.
Root Causes: This problem is likely to occur when [12]: (1) learning rate is too high/low. (2) there is a large negative bias. (3) improper weight or bias initialization.

3.2.2
Symptom #2 Saturated Activation. The Saturated Activation symptom takes place when the input to the logistic activation function (e.g., tanh or sigmoid) reached either a very large or a very small value [19]. At the saturated point, the function results would equal zero or be close to zero and, thus, leading to no weight updates. Our experiments show [6] that the behavior of sigmoid and tanh have a minimum saturated point at x=-5 and a maximum saturated point at x=5. Previous work showed that the saturated function affects the network's performance and makes the network difficult to train [19,40].
Root Causes: This problem is likely to occur when [15]: (1) the input data are too large or too small; (2) improper weight or bias initialization; (3) learning rate is too high or too small.

3.2.3
Symptom #3 Exploding Tensor: The Exploding Tensor symptom takes place when the tensors' values become too large, leading to numerical errors in a feed-forward stage. For example, if the weight or output layer grows exponentially more than expected, becoming either infinite or NaN (not a number). Eventually, this problem causes a numerical error, making it impossible for the model to learn.
Root Causes: This problem is likely to occur when [16,24]:(1) the learning rate is too large; (2) there exist improper weight or bias initialization, or improper input data.

Symptom #4 Accuracy Not Increasing & Symptom #5
Loss Not Decreasing. Both symptoms Accuracy Not Increasing and Loss Not Decreasing are very similar. The Accuracy Not Increasing symptom takes place when the accuracy of a target model is not increasing for N steps, but instead, it is decreasing or fluctuating during training. While for the Loss Not Decreasing symptom, the loss metric is the one that is not decreasing for N steps but is fluctuating. These behaviors indicate that the network will not achieve high performance. These symptoms are often caused by the incorrect selection of DNN hyperparameters [32], such as loss function, activation function for the last layer, learning rate, optimizer, or batch size.
Root Causes: This problem is likely to occur when [16,24]: (1) there exist improper training data; (2) the number of layers is too The procedure stores the value for a given number of steps (N = 5). Then it compares the value for the current step with the mean value stored in for previous (N = 5) steps. The evaluation takes place for every given number of steps. The procedure returns True if the value is not changing, otherwise False.

S3
SaturatedActivation () Input of Activation Function T | F The procedure detects if the tanh or sigmoid activation functions, or other logistic functions are becoming saturated. It does so by checking if their input has reached either a maximum or minimum value. Saturated functions' derivatives would be equal to zero at those points. The procedure counts the activity of a close or greater node than to the (Max_Threshold = 5) or less than (Min_Threshold = -5) of the activation function; If the percentage of total activity nodes is greater than the (Threshold_Layer = 0.5) percent of the nodes are saturated, the procedure returns True, otherwise False.

S4
DeadNode () Relu Output T | F This procedure takes the output of Rectified Linear Unit (ReLU) activation function as input, then computes how many inactive nodes dropped below (Threshold = 0.0). If the percentage of inactive nodes is greater than (Layer_Threshold = 0.7) it returns True, otherwise False.

S5
OutofRange () Output of last layer T | F The procedure detects if the activation function's output is becoming out of range for the labeling training dataset Y. To that end, it finds the range (maximum and minimum) of the activation function's output. Then compare it with Y labeling data. If the value is out of the boundary, the procedure returns True, otherwise False.
This procedure checks the variance of weight inputs across layers to determine if a neural network has been poorly initialized. The procedure checks if the variance of weights per layer is equal or very close to 0 (Min_Threshold = 0.00001), or if it exceeds the (Min_Threshold = 10), the procedure returns True. Otherwise, False.

C3
TuneLearn () Learning rate, Weight, and Δ Weight L | H The procedure evaluates the learning rate heuristically by computing the ratio of the norm of the gradient weight to the norm of weight for each layer. This ratio should be somewhere around (Learn_Threshold = 1e-3). If it is lower than (Learn_Threshold = 1e-3), then the learning rate might be too Low. If it is higher than (Learn_Threshold = 1e-3), the learning rate is likely too High.
This table is showing all the functionality of the procedures. T |F indicates the procedure return True | False respectively. L|H indicates the procedure return Low| High respectively. We borrowed these methods from existing literature [1, 10, 16,32,43] large/small; and (3) the learning rate is very high/low; and (4) there exist incorrect activation functions.

3.2.5
Symptom #6 Unchanged Weight. The Unchanged Weight symptom takes place when the DNN weights do not have a noticeable influence on the output layers. This behavior leads to unchanging parameters and network stacks, which further prevents the model from learning [15,39]. Root Causes: This problem is likely to occur when [15,39]: (1) learning rate is very low; (2) the optimizer is incorrect; (3) there exist incorrect weights initialization; and (4) there exists incorrect loss/activation at the last layer.
3.2.6 Symptom #7 Exploding Gradient. This problem occurs during the back-propagation stage. In it, gradients are growing exponentially from the last layer to the input layer, which leads to non-finite values, either infinite or NaN (not a number). This issue makes learning unstable and sometimes even impossible. Consequently, updating the weights becomes very hard, and the training model ends up with a high loss or very low accuracy values.
Root Causes: This problem is likely to occur when [16,24]: (1) the learning rate is very high; (2) there is an improper weight or bias initialization; (3) there are improper data input; and (4) the batch size is very large.
3.2.7 Symptom #8 Vanishing Gradient. The Vanishing Gradient problem occurs during the backward stage. When computing the gradient of the loss concerning weights using partial derivatives, the value of the gradient turns out to be so small or drops to zero. The problem causes major difficulty if it reaches the input layer, which will prevent the weight from changing its value during training. Since the gradients control how much the network learns during training, the neural network will end up without contributing to the prediction task or leading to poor performance [36,43].
Root Causes: This problem is likely to occur when [25]: (1) the network has too many layers; (2) the learning rate is low; (3) the hidden layers improperly used Tanh or Sigmoid; and (4) there exists the incorrect weight initialization problem.

Detecting Failure Symptoms
In Table 2 from Method S1 to S8, we describe the failure symptoms discussed in Section 3.2, using its name, input/output, and the detection procedure. Algorithm 1 shows an example of a dynamic analysis procedure, which DeepDiagnosis uses to detect failure symptoms during training (Table 2 Description). Also, the Algorithm 1 reports failure locations, such as in which layer and phases (i.e., feed-forward and backward propagation). In case a failure is detected, the algorithm will trigger the Mapping() procedure to identify the location in the original DNN source code. By doing so, it will localize the bug and determine the optimal fix.
At line 1, Algorithm 1 iterates over the training epochs, with the training dataset divided into batches. Line 3 shows the division of the training dataset into a mini-batch. On lines 2-28, the algorithm runs one batch of the training dataset before updating the internal model parameters. The neural network can be divided into two stages: First, the forward stage, in which the algorithm performs dynamic analysis and symptom detection, including Numerical Error, Dead node, Saturated Activation, and Out of Range, at lines 4-12. Second, the backward stage, in which the algorithm performs dynamic analysis to detect additional symptoms, such as Numerical Error, Vanishing Gradient, and Unchanged weight at lines 23-28.
3.3.1 Feed-forward stage. At lines 5 & 6 of the Algorithm 1, it computes the output of a feed-forward before and after applying the activation function. At line 7, it invokes the ExplodingTensor() procedure (S1 in Table 2) to determine if the output contains a numerical error obtained from the output value before/after the applying activation function, respectively. If there is an error, the algorithm reports the NS message as shown in Table 5. Next, it invokes the Mapping() procedure from the decision tree in Figure 2 by providing the symptom (NS), location, stage (FW), and layer (L). The decision tree returns the best actionable fix for the model (see Section 3.4 for more details).
At line 8, the Algorithm 1 invokes the UnchangeWeight() (S2 in Table 2) procedure to detect whether the output before/ after applying the activation function is no longer changing across steps. If the procedure indicates that the value does not change for N iterations, we follow [39] and set N=5. The UnchangeWeight() procedure can be applied either to the output before/after the activation function. The algorithm reports the message UCS, as shown in Table 5. At line 9, the Algorithm invokes the SaturatedActivation () procedure (S3 in Table 2) for the layer that has a logistic activation function (i.e., tanh or sigmoid) to determine if the layer is becoming saturated. This procedure takes two arguments, the value before applying the activation function (V_1) and the name of the activation function (V_2.name). If the procedure determines that the layer is saturated, the algorithm reports the message SAS as shown in Table 5.
At line 10, the Algorithm 1 invokes the DeadNode() procedure (S4 in Table 2) to check the layers that use the Rectified Linear Unit (ReLU) activation function. The goal is to determine if the output after applying the activation function has dropped below a threshold [43]. This procedure is invoked only after applying the activation function. The algorithm reports the message DNS as shown in Table 5 when the error is detected. Similarly, at line 11, it invokes the OutofRange() procedure (S5 in Table 2) in the last layer. The goal is to determine if the developer has chosen the correct activation function. The algorithm reports the message ORS as shown in Table 5 if the error is detected.
In lines 13 & 15 the algorithm interprets and validates how well the model is doing by computing the loss and accuracy metrics, Change the activation function MSG2 4 Change the learning rate MSG3 5 Change the initialization of weight MSG4 6 Change the layer number MSG5 7 Change the optimizer MSG6  Table 2) and passes Δ Weight to check if the gradients become extremely small or close to being zero. In the same way, at line 26, the algorithm can determine if there is a numerical error in the Weight or the gradient weight in each layer by invoking the ExplodingTensor() procedure (S1 in Table 2). The backpropagation algorithm works if the Weight is updated using the gradient method and the loss value keeps reducing, to check if the backpropagation works effectively. In the backward propagation, we also invoke the UnchangeWeight() procedure (S2 in Table 2) to detect whether the weight or Δ Weight is no longer changing across steps. If any procedure decides that there is an issue, then the algorithm will return a message to indicate the type of symptom as shown in Table 5, L represents a faulty layer number. Then the algorithm invokes Mapping() and passes the symptom, location, and layer to find the best actionable change to fix the issue in the model. Finally, if the algorithm did not detect any type of symptom, it will terminate after finishing the training at line 29 and print a message indicating that there is no issue in the model (CM).

Mapping Symptoms to Location fix
Decision Tree: The main goal of this step is to mitigate manual effort and reduce the time for diagnosing and fixing bugs. To that end, the Mapping() procedure in Algorithm 1 provides fix suggestions based on the detected failure symptoms. Figure 2 shows a representation of the Decision Tree which the Mapping() procedure uses to provide a fix recommendation.
The Decision Tree consists of 24 rules, which corresponds to decision paths. Each rule provides a mapping from failure symptoms and detected locations to actionable changes. The tree defines a binary classification rule which maps instances in the format problem (Symptom, Location, Layer) into one of seven classes of changes ( Table 4). The root node represents the problem, orange nodes the symptoms, blue nodes the locations, gray nodes the layer type, green nodes, the conditions, and red nodes the actionable changes. Table 3 shows the methods Data(), Weight() and Learn(), which are used to compute the conditions. Each Decision Tree instance maps a path from the root to one of the leaves.
For example, assume that a developer wants to check the code in Listing 1. To that end, the developers can use the Algorithm 1 to verify the model. The algorithm invokes the Mapping() procedure (line 26) by passing the symptom NS, location, stage BW (backward), and layer (7). This procedure traverses the path under the NS node in the Decision Tree (Figure 2). Since the problem occurred in the BW stage, the algorithm takes the right path to satisfy the condition. Then, it verifies the layer type (7). Since it finds an issue in the layer, the procedure returns the message MSG2 -Change the activation function (Table 4).

Heuristics:
We developed a set of heuristics based on the root causes (see Section 3.2). There are three main root causes: (1) Data Preparation; (2) Parameter Tuning; and (3) Model Architecture. For Data Preparation, the algorithm checks if the data is normalized (C1 -ImproperData() in Table 3). For Parameter Tuning, our approach checks if the hyperparameters (such as learning rate) were assigned correctly. Also, to check if the weights were initialized correctly, the algorithm invokes the WeightInitialization(). The TuneLearn() procedure verifies whether the learning rate is very high or very low (C2, and C3 in Table 3, respectively). For model architecture, the algorithm searches for a relation between the location and the stage of the symptom. It performs this step to pinpoint which APIs are being misused in the model (e.g., loss, activation function).
We collected the root causes for each symptom from previous work [19,20,29,35] (more details in Section 3.2). To arrive at a possible fix for a given symptom, we choose the most frequent root cause. We follow this approach as our findings show that changes in the order we check for the possible root causes do not affect the results, only on the total time to arrive at a solution. For example, assume that a model has the Dead Node symptom. In terms of frequency, improper data tends to happen more often than weight and learning rate. If the three root causes are correct, our approach checks the model architecture, which is the least common in this case. Thus arriving at an improper activation function as the root cause of this symptom.

EVALUATION
In the evaluation, we answer the following research questions: 4.1.2 Benchmark. In total, we collected 548 models from prior work [7, 33,43]. From these, we removed 104 RNN models, as our approach does not support them. The resulting 444 models are composed of 53, which are known to have bugs from [7, 33]. We refer to these 53 models SGS benchmark as they come from StackOverflow, GitHub, and Schoop et al. [33]. Also, the 391 models from [43] are in the compiled *.h5 format. The remaining 391 models are divided into two sets. In particular, the first with 188 correct modelswithout any known bugs -and the second with 203 buggy modelswith bugs.
Most machine learning developers share the source code or the trained weights of their models in *.h5 format. To allow others to improve the understanding of how a model operates and inspect it with new data, we implemented the Extractor tool [6]. It extracts the DNN source code from a *.h5 file. The Extractor follows three steps to generate the Keras source code: first, it saves the model's layer information to a JSON file. Then, it generates the Abstract Syntax Tree (AST) from the JSON file. Finally, it converts the AST to the source code.
To build the ground truth for the SGS benchmark, we manually reviewed all models and their respective answers, from Stack Overflow, and commits, from GitHub. We perform this verification process to determine the exact bug location and its root causes. For the remaining 391 models -203 buggy models and 188 not buggy models, we used our Extractor to generate the source code for each model before/after performing a fix; we used the difflib [18] module to generate the diff file from the fixed model. We use the diff to locate the changes in the model, thus locating the root causes and the actual location of its corresponding fix. We consider a model successfully repaired if its accuracy has improved after the fix. Table 6 shows the summarized evaluation results of the following approaches: UMLUAT [33], DeepLocalize [39], AUTOTRAINER [43], and our approach DeepDiagnosis. Please refer to the reproducibility repository  Table 2 detects the failure symptoms. Under each approach, the "Yes" and "No" status indicates whether it has successfully reported the target problem. Also, the "-" status denotes if the approach does not yet support the target problem. Lastly, we use method ID in Table 2 to indicate which procedure is used to detect the failure symptom. Table 7 summarizes the analysis results from four approaches using benchmarks collected from three different sources [33,39,43]. The second column (Total) lists the total number of buggy models for each dataset. To compare our approach with previous approaches, we reported Time, in seconds, the average time for each tool takes to report its results for each dataset. The columns Identify Bug (IB) shows how many each approach successfully identifies the bug from each dataset. Our approach is capable of handling eight types of symptoms with different types of datasets using different types of model architectures. Table 8 shows the number of symptoms detected from different types of datasets. DeepDiagnosis (DD) has correctly identified 46 out of 53 buggy models from the SGS benchmark. DD correctly reported the fault location for 34 models and the failure symptoms for 37 models. Also, DD correctly identified the actionable changes for 28 out 53 faulty models. Lastly, DD identified 138 out of the 203 buggy models from the AUTOTRAINER dataset, correctly reporting fault location, failure symptoms, and actionable changes.

RQ1 (Validation) and RQ2 (Comparison)
DeepLocalize (DL) [39] identified 45 out of the 53 models from the SGS benchmark and indicated fault locations for 26. It reported symptoms for only 23 models, but it cannot provide any suggestions to fix these faults. Regarding the AUTOTRAINER dataset, DL identified 191 out of the 203 buggy models and correctly reported their fault location. However, DL did not provide any suggestions for fixing those models. Lastly, DL can only detect bugs related to numerical errors.
AUTOTRAINER (AT) [43] For the 53 models (SGS benchmark), AT identified 24 buggy models. Out of these, AT successfully reported symptoms for only 8. AT was only able to repair 16 models. DD can handle more varieties of semantically related errors than AT, as shown in Table 6. Please refer to [43] for AT's evaluation results while analyzing its dataset.
UMLUAT (UM) [33] identified 26 buggy models out of the 53 from the SGS benchmark and found the fault locations for 3. Also, UM reported the symptoms for 17 models and provided the location fix for 15 out of 53. UM correctly identified models and reported possible fix solutions to problems from 72 out of 203 buggy models of the AUTOTRAINER dataset. UM only supports classification problems, while DD supports additional types, such as regression and classification.  To evaluate the approaches' overall performance, we collected their total execution time while analyzing the benchmarks. Figure 3 shows the results. UM, DL, AT, and DD require on average 46. 16 DD' runtime overhead is mainly due to its online dynamic analysis. DD runs its dynamic analysis on the internal parameters of the neural networks, such as the changes of weights and gradients, during the training phase. DD is the most efficient for Stack Overflow and Schoop et al.'s model, and is slower than UM on the GitHub models. The reason is that DD collects more information than UM during training and checks additional types of error conditions. DD is faster than AT on all benchmarks except for the Blob and Circle datasets. That is because AT checks the target model after finishing the training phase. DD requires additional time because it validates the model at the end of each epoch during training, and the number of epochs for these models are between 200 to 500.

RQ3 (Limitation)
Out of 52 programs, our technique failed to identify faults in 6 and localize faults in 18. DD failed to report symptoms for 15 programs and failed to provide the location of fix for 24 (Tables 2 and 6). In the following, we provide a few examples failed fault localization.
Our technique does not yet support model with fit_generator() instead of fit() function. fit_generator() is used for processing a large training dataset that is unable to load into the memory [13]. In the future, we plan to cover more APIs (such as fit_generator()).
Both #47 (B3 (C10)), and #53 (B3 (C10)) programs are related to checking validation accuracy [33]. The model splits the train data into training and validation data, and then provide the validation data by passing validation_data=(x_val, y_val) into the fit() method. The buggy model reported high accuracy for the validation dataset. There may exist an overlap between training data and validation data. But our approach would not indicate any symptom, as it does not support problems related to training and validation.
Both #43 (A2 (C10)), and #49 (A2 (C10)) programs are related to the dropout rate in the Dropout layer [33]. The idea of the dropout is to remove certain percentage of neurons during iterations to prevent the overfitting. The buggy model sets a high dropout rate = 0.8 which is more than the acceptable rate 50%. Our approach is not able to provide a correct suggestion to fix the model. In our future work, we plan to investigate more hyperparameters such as the batch size, epoch, and dropout rate to handle the above models.
DD supports deep learning models of various structures, including convolutional neural networks (CNNs) and fully connected layers. But, Recurrent Neural Networks (RNNs) are not supported by our current reference implementation. Developers can extend our DD to support RNNs and other architectures.
UM only supports classification problems, in which the last layer is softmax. Otherwise, it reports false alarms. DL only supports numerical problems, and it does provide any suggestions on how to fix a detected problem. AT supports classification problems and does    not support problems in the model architecture (i.e., loss function, activation function at last layer, and some APIs (e.g.,fit_generator())).
In terms of efficiency, AT takes longer to find a fix, as it tries all possible solutions until arriving at the correct one. In case it does not find an improvement, it marks the problem as unsolvable.

RQ4 (Ablation)
The "Ablation" column of Table 6 shows which procedure in Table 2 is used to report the symptom in each buggy model for SGS dataset. We found that ExplodingTensor () detects 23 buggy models, SaturatedActivation () detects 7, DeadNode () reports 5, OutofRange () detects 5, UnchangeWeight () finds 2, InvalidAccuracy () detects 2, AccuracyNotIncreasing (), and VanishingGradient () reports only one buggy model. Table 8 shows dataset names, and columns contain the number of symptoms, which were detected successfully by the corresponding procedure in Table 2. From Table 8, we found that ExplodingTensor () detects 73 buggy models, VanishingGradient () detects 32, UnchangeWeight () finds 25, AccuracyNotIncreasing () 22, DeadNode () reports 13, SaturatedActivation () detects 12, Out-ofRange () detects 5 , and InvalidAccuracy () reports only two buggy models. Although the incorrect DNN models related to parameters and structures often manifest as numerical errors during training, DD provided further reasoning and categories of causes using these procedures, which can help quickly fix the bugs. Our study also found that data preparation is a frequently occurring issue and thus the ImproperData() procedure is frequently invoked. SGS benchmark does not have a very deep model that contains many layers. Thus we did not use VanishingGradient () detector very frequently.
On the other hand, VanishingGradient () is invoked very frequently in AUTOTRAINER models, because this dataset has many layers using sigmoid and tanh as activation functions. However, when N layers use a Logistic activation function (like sigmoid or tanh), N small derivatives are multiplied together. Thus, the gradient decreases exponentially and propagates down to the input layer.

Results Discussions
We compared and contrasted three approaches [32,39,43] against our approach (DD). From Table 7, we found our approach detected more problems in the SGS dataset than AUTOTRAINER. Also, it detected fewer problems in AUTOTRAINER dataset than the AT approach. The reason is that our approach only reported the problem and solution if it detected one of 8 symptoms. On the other hand, AT inspects the model based on the training accuracy threshold [43]. For our evaluation, we used 188 normal models from [43]. 78 are MNIST, 35 are CIFAR-10, 36 are Circle, and 39 are Blob. UM reported the message: "<Warning: Possible overfitting>" for 68 out 78 MNIST models. It reported the following message: "[<Error: Input data exceeds typical limits>]" for 35 out 35 CIFAR-10 models, because the training data is not in the range [-1, 1]. DL reported message: "MDL: Model Does not Learn" for 4 out 34 Circle models and 16 out 39 Blob models. For all MNIST and CIFAR-10 models, DL reported different messages. AT checks if a model has training accuracy less than or equal to the threshold of 60%. To make a fair comparison between the approaches, we changed the training accuracy threshold to 100%. AT reported different symptoms for 10 out of 36 Circle, 5 out of 39 Blob models, and 2 models with problems out of the 78 MNIST models. Our approach reported one saturated symptom for 36 Circle, which is not supported in AT, reported 8 symptoms -6 "saturated activations" and 2 the "accuracy is not increasing. " For the MNIST model, our approach reported 37 symptoms -35 "dead nodes" and one is a "numerical problem;" we investigated this model and found its accuracy is 20%. For CIFAR-10 models, DD reported 21 models with "dead node" out of 35 models. All detailed experiment results are publicly available [6].

Summary
DD significantly outperformed the baselines UM, DL, and AT in the SGS dataset (Tables 6 and 7). In particular, identified 46 out of 53 buggy models, correctly performed fault localization in 34 models, and reported symptoms for 37 of those. DD also provided a location to fix 28 out of 53 faulty models. Regarding total analysis time, DD outperformed DL and AT in benchmarks. As DD does not require the training phase to finish to detect bugs. Also, DD uses a Decision Tree (Figure 2) approach to reduce the search space when mapping symptoms to their root causes.
Furthermore, DD is more comprehensive than prior work, as it supports several varieties and semantically related errors in classification and regression models. Also, DD supports 8 failure symptoms, while prior approaches support fewer (in Section 3).
Finally, DD does not support some APIs (e.g., fit_generator()) as we consider problems related to hyperparameters, for example, epoch, batch size, and dropout rate, as out of scope.

THREATS TO VALIDITY
External Threat: We have collected 53 real-world buggy DNN models from Stack Overflow, GitHub and 496 models from prior work [33,39,43]. These models cover a variety of failure symptoms and location to perform fixes; however, our dataset may not include all types of DNN APIs and their parameters. To mitigate the threat of behavior changes caused by the Extractor tool, used to extract the source code from the 496 models [43]. We have verified the accuracy of each model before and after their conversion. In terms of execution time, different hardware configurations may offer varying response times. We mitigated this threat by executing our experiments several times and calculated their averages.
Internal Threat: When implementing Algorithm 1, Decision Tree (Figure 2), and Tables 2 and 3, we used the parameters defined by prior works [1, 10, 16,32,43]. These selected values may not work for some unseen examples. To mitigate this threat, we have validated these selected parameters against our benchmarks collected from a diverse set of sources [33,39,43]. For each of these benchmarks, our selected parameters work consistently well. Although we have carefully inspected our code, our implementation may still contain some errors. We manually constructed ground truths regarding fault location, failure symptoms, and location to fix for all the buggy models based on the data from the previous research [33,39,43]. This process may have introduced errors.

RELATED WORK
Fault localization in Deep Neural Networks: The recent increase in the popularity of deep learning apps has motivated researchers to adapt fault localization techniques to this context. With the intent of validating different parts of DL-based systems and discovering faulty behaviors. The goal of fault localization is to identify suspicious methods and statements, to isolate the root causes of program failures, and reduce the effort of fixing the fault [31]. Wardat et al. [39] presented an automatic approach for fault localization called DeepLocalize. It performs dynamic analysis during training to determines if a target model contains any bugs. It identifies the root causes by catching numerical errors during DNN training. While DeepLocalize focuses on identifying bugs and faults based on numerical errors, DeepDiagnosis aims to perform fault localization beyond that scope. Furthermore, our approach can report symptoms and provide actionable fixes to a problem.
DEBAR [44] is a static analysis approach that detects numerical bugs in DNNs. DEBAR uses two abstraction techniques to improve its precision and scalability. DeepDiagnosis uses dynamic analysis to localize faults and report symptoms of a model during training. In contrast, DEBAR uses a static analysis approach to detect numerical bugs with two abstraction techniques.
Schoop et al. [33] proposed UMLUAT, a user interface tool to find, understand and fix deep learning bugs using heuristics. It enables users to check the structure of DNN programs and model behavior during training. Then, it provides readable error messages to assist users in understanding and fixing bugs. Section §4 shows the comparison between UMLUAT [33] and DeepDiagnosis. DeepDiagnosis is more comprehensive, efficient, and effective than UMLAUT, which only supports classification models.
DeepFault [11] is an approach that identifies suspicious neurons of a DNN and then fixes these errors by generating samples for retraining the model. DeepFault is inspired by spectrum-based fault localization. It counts the number of times a neuron was active/inactive when the network made a successful or failed decision. It then calculates a suspiciousness score such as the spectrum-based fault localization tool Tarantula. In contrast, DeepDiagnosis focuses on identifying faults and reporting different types of symptoms for structure bugs.
Bug Repair in Deep Neural Networks: Zhang et al. [41] proposed Apricot, an approach for automatically repairing deep learning models. Apricot aims to fix ill-trained weights without requiring additional training data or any artificial parameters in the DNN. MODE [26] is a white-box approach that focuses on improving the model performance. It is an automated debugging technique inspired by state differential analysis. MODE can determine whether a model has overfitting or under-fitting problems. Compared with MODE and Apricot, which focus on training bugs (e.g., insufficient training data), DeepDiagnosis focuses on structure bugs (e.g., activation function misused).
Zhang et al. [43] introduced AUTOTRAINER, an approach for fixing classification problems. Zhang et al. define five symptoms, and provide a set of possible solutions to fix each one. Once AUTO-TRAINER detects a problem, it tries the candidate solutions, one by one, until it addresses the problem. If none of the solutions fix the problem, it reports a failure message. The evaluation used six popular datasets and showed that AUTOTRAINER detects and repairs the models based on a specific threshold. AUTOTRAINER was able to improve the accuracy for all repairing models on average 47.08%. DeepDiagnosis analyzes the model's source code during the training phase to localize the bug. DeepDiagnosis supports eight symptoms, while AUTOTRAINER supports five. DeepDiagnosis does not perform automated fixes, but it provides actionable recommendations that developers can follow. AUTOTRAINER tries all possible strategies in its search space to fix a problem and outputs whether or not the fix was successful. In contrast, DeepDiagnosis uses a decision tree to reduce the solution search space, thus saving time and computational resources. In summary, the goals of DeepDiagnosis and AUTOTRAINER are different; DeepDiagnosis focuses on fault localization while AUTOTRAINER on automatically repairing a model.

CONCLUSIONS AND FUTURE WORK
This paper introduces a dynamic analysis approach called DeepDiagnosis that a non-expert can use to detect errors and receive useful messages for diagnosing and fixing the DNN models. DeepDiagnosis provides a list of verification procedures to automatically detect 8 types of common symptoms. Our results show that DeepDiagnosis can successfully detect different types of symptoms and report actionable changes. It outperforms the state of the art tool such as UMLUAT and DeepLocalize, and it is faster than AUTOTRAINER for fault localization and provide suggestions to fix the issue. In the future, we plan to expand this prototype to handle more types of models and failure symptoms, and also automatically fix the bugs.