{'fraction_non_alphanumeric': 0.03129615952732644, 'fraction_numerical': 0.00947808961102905, 'mean_word_length': 4.911770056394397, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Syntax-Guided Program Reduction for Understanding Neural Code Intelligence Models

Neural code intelligence (CI) models are opaque black-boxes and offer little insight on the features they use in making predictions. This opacity may lead to distrust in their prediction and hamper their wider adoption in safety-critical applications. Recently, input program reduction techniques have been proposed to identify key features in the input programs to improve the transparency of CI models. However, this approach is syntax-unaware and does not consider the grammar of the programming language. In this paper, we apply a syntax-guided program reduction technique that considers the grammar of the input programs during reduction. Our experiments on multiple models across different types of input programs show that the syntax-guided program reduction technique is faster and provides smaller sets of key tokens in reduced programs. We also show that the key tokens could be used in generating adversarial examples for up to 65% of the input programs.


Introduction
A neural code intelligence (CI) model is a deep neural network that takes a program as input and predicts certain properties of that program as output, e.g., predicting method name [3], variable name [5], or type [12] from a program body. Unfortunately, these models are opaque black-boxes and it is hard to interpret and debug the results of these models. Recently, there is an increasing interest in probing CI models to improve their transparency and identify and understand their potential flaws. For instance, recent studies suggest that state-of-the-art CI models do not always generalize to other experiments [13,14]; they heavily rely on specific individual tokens [11,17] or structures [15,19], are prone to learn noise [18] or duplication [1], and are often vulnerable to semantic-preserving adversarial examples [16,20,25].
Neural CI models, as in other neural models, represent an input program as continuous distributed vectors that are computed after training on large volumes of programs.
It makes understanding what input features a model has learned a very challenging task. For example, Code2Vec model [8] learns to represent an input program as a single fixed-length high dimensional embedding, however, it is difficult to understand the meaning or characteristics of each of these dimensions. Attention-based approaches are commonly applied to find important code elements in a program. For example, Bui et al. [9] attempt to identify relevant code elements by perturbing statements of the program and combining corresponding attention and confidence scores. However, the attention-based approach poorly correlates with key code elements.
Several studies have already been conducted to find relevant input features in models' inferences. Allamanis et al. [3] use a set of features from programs and show that extracting relevant features that capture global context is essential for learning effective code context. Rabin et al. [19] attempt to find key input features of a label by manually inspecting some input programs of that label. However, this manual inspection does not scale to large datasets with many labels. Suneja et al. [23] and Rabin et al. [17] use input program reduction techniques, based on Delta Debugging [26], to find minimal inputs that preserve the model's prediction, hence finding key tokens in the program with respect to the prediction. However, these approaches are syntax-unaware and can create a large number of invalid programs as they do not follow the grammar of the programming language during the reduction, which in turn can hinder the identification of the key tokens.
In this paper, we apply a syntax-guided reduction technique, called Sivand-Perses, to remove irrelevant parts from an input program. Given a CI model and an input program, our approach adopts Perses [22] to reduce the input program while preserving the model's prediction. The approach continues reducing the input program as long as the model maintains the same prediction on the reduced program as on the original program. As the syntax-guided technique follows the syntax of the programming language, it will always generate valid input programs. Using the syntax information also can improve the performance of the reduction, as the reduction only explores the space of syntactically valid programs. The main insight is that, by reducing some input programs of a target label, we may better understand what key input features a CI model learns from the training dataset.
An experiment with two CI models and four types of input programs suggests that the syntax-guided Sivand-DD outperforms the syntax-unaware alternative Sivand-DD proposed in [17]. While Sivand-Perses always generates valid programs, Sivand-DD generates only around 10% valid programs. On average, Sivand-Perses removes 20% more tokens, takes 70% fewer reduction steps, and spends half of the reduction time compared to Sivand-DD for reducing an input program. Furthermore, our results show that we can find key input features by reducing input programs using Sivand-Perses, which can provide additional explanation for a prediction and highlight the importance of input features in reduced programs by triggering 10% more mispredictions with 50% fewer adversarial examples.
Contributions. This paper makes the following contributions: • We propose a syntax-guided program reduction approach, Sivand-Perses, for the identification of key features in neural code intelligence models. • We evaluate the performance of Sivand-Perses and compare it with the state-of-the-art technique on two CI models for the code summarization task. • We observe that different program reduction techniques can provide complementary explanations of a specific prediction for better understanding.

Related Work
There has been some work in the area of code intelligence that focuses on the understanding of what relevant features a black-box model learns for correct predictions. While some works [11,13,14,16,17,20,23,25] study the reliance of models on specific features, many works [3,9,17,19,23,24] focus on finding relevant features for explaining models' prediction.

Learning Representation of Source Code
An input program is usually represented as vector embeddings for processing and analyzing by neural models. Allamanis et al. [2] introduced a framework that processed token sequences and abstract syntax trees of code to represent the raw programs. Alon et al. [8] proposed an attention-based neural model that uses a bag of path-context from abstract syntax tree for representing any arbitrary code snippets. Allamanis et al. [5] constructed data and control flow graphs from programs to encode a code snippet. Hellendoorn et al. [12] proposed an RNN-based model using sequence-to-sequence type annotations for type suggestion. There are some surveys on the taxonomy of models that exploit source code analysis [4,21]. Chen and Monperrus [10] also provide a survey that includes the usage of code embeddings based on different granularities of programs. However, these models are often black-box and do not provide any insight on the meaning or characteristic of learned embeddings. What features or patterns these embeddings represent are largely unknown. In this work, we extract key input features that a model learns for predicting a target label as an explanation of learned embeddings.

Reliance on Specific Features
Models often learn irrelevant features, simple shortcuts, or even noise for achieving target performance. Compton et al. [11] show that the code2vec embeddings highly rely on variable names and cannot embed an entire class rather than an individual method. They investigate the effect of obfuscation on improving code2vec embeddings that better preserves code semantics. They retrain the code2vec model with obfuscated variables to forcing it on the structure of code rather than variable names and aggregate the embeddings of all methods from a class. Following the generalizability of word embeddings, Kang et al. [13] assess the generalizability of code embeddings in various software engineering tasks and demonstrate that the learned embeddings by code2vec do not always generalize to other tasks beyond the example task it has been trained for. Rabin et al. [16] and Yefet et al. [25] demonstrate that the models of code often suffer from a lack of robustness and are vulnerable to adversarial examples. They mainly introduce small perturbations in code for generating adversarial examples that do not change any semantics and find that the simple renaming, adding or removing tokens changes model's predictions. Suneja et al. [23] uncover the model's reliance on incorrect signals by checking whether the vulnerability in the original code is missing in the reduced minimal snippet. They find that model captures noises instead of actual signals from the dataset for achieving high predictions. Rabin et al. [17] demonstrates that models often use just a few simple syntactic shortcuts for making prediction. Rabin et al. [18] later show that models can fit noisy training data with excessive parameter capacity and thus suffer in generalization performance. As models often learn noise or irrelevant features for achieving high prediction performance, the lack of understanding of what input features models learn would hinder the trustworthiness to correct classification. Such opacity is substantially more problematic in critical applications such as vulnerability detection or automated defect repair. In this work, we extract key input features for CI models in order to provide better transparency and explanation of predictions.

Extracting Relevant Input Features
Several kinds of research have been done in finding relevant input features for models of source code. Allamanis et al. [3] exhibit that extracting relevant features is essential for learning effective code context. They use a set of hard-coded features from source code that integrate non-local information beyond local information and train a neural probabilistic language model for automatically suggesting names. However, extracting hard-coded features from source code may not be available for arbitrary code snippets and in dynamically typed languages. Bui et al. [9] propose a code perturbation approach for interpreting attention-based models of source code. It measures the importance of a statement in code by deleting it from the original code and analyzing the effect on predicted outputs. However, the attention-based approach often poorly correlates with key elements and suffers from a lack of explainability. Rabin et al. [19] attempt to find key input features of a label by manually inspecting some input programs of that label. They extract handcrafted features for each label and train simple binary SVM classification models that achieve highly comparable results to the higher dimensional code2vec embeddings for the method naming task. However, the manual inspection cannot be applied to a large dataset. Wang et al. [24] recently propose a mutate-reduce approach to find key features in the code summarization models considering valid programs. Suneja et al. [23] and Rabin et al. [17] apply a syntax-unaware program reduction technique, Delta Debugging [26], to find the minimal snippet which a model needs to maintain its prediction. By removing irrelevant parts to a prediction from the input programs, the authors aim to better understand key features in the model inference. However, the syntax-unaware approach creates a large number of invalid programs during the reduction as it does not follow the syntax of programs, thus increases the total steps and time of reduction. In this work, we apply a syntax-guided program reduction technique that overcomes the overhead raised by the syntax-unaware technique.

Design and Implementation
This section describes our approach of extracting input features for code intelligence (CI) models by syntax-guided program reduction. We use Perses [22] as the syntax-guided program reduction technique in our study. We first provide an overview of how Perses works and then describe how we adopt it in the workflow of our Sivand-Perses approach.
Perses Sun et al. [22] have proposed the framework for syntax-guided program reduction called Perses. Given an input program, the grammar of that programming language, and the output criteria, Perses reduces the input program with respect to the grammar while preserving the output criteria. It mainly follows the steps below: • It first parses the input program into a parse tree by normalizing the definition of grammar. • Then it traverses the tree and determines whether a tree node is deletable (e.g. whether it follows the grammar and preserves the output criteria). If yes, it prunes the sub-tree from that node and generates a valid reduced program, else it ignores that node and avoids generating invalid programs. Thus, in each iteration of reduction, it ensures generating syntactically valid program variants that preserves the same output criteria. • Next, the deletion of one node may enable the deletion of another node. Therefore, Perses is repeatedly applied to the reduced program until no more tree nodes can be removed -a process known as fixpoint mode reduction. • The final reduced program is called 1-tree-minimal, and any further attempts to reduce the program would generate an invalid program or change the output criteria.
For supporting a programming language data, the syntaxguided technique leverages knowledge about program syntax for avoiding generating syntactically invalid programs. We integrate Perses as a black-box framework into Sivand-Perses for extracting input features of CI models.
Workflow. Figure 1 depicts a high-level view of the workflow in Sivand-Perses. Given a set of input programs, our approach reduces each input program using Perses while preserving the same prediction by the CI model. The approach removes irrelevant parts from an input program and keeps the minimal code snippet that the CI model needs to preserve its prediction. The main insight is that, by reducing some input programs of a target label, we can identify key input features of the CI model for that target label. Our approach follows the steps below: • Given an input program and a CI model , our approach first records the prediction (i.e., predicted method name) given by the CI model on the input program , such as = ( ). • Using Perses, we then generate a candidate reduced program ′ by removing some nodes from the tree of the input program , such as ′ = ( ). Figure 1. Workflow of Sivand-Perses approach.
• If the candidate reduced program ′ does not hold the same prediction by the CI model (i.e., ≠ ( ′ )), we reject this candidate program and create another candidate program by removing some other nodes from the tree of the input program. • If the candidate reduced program ′ preserves the same prediction by the CI model (i.e., = ( ′ )), we continue reduction and iteratively search for the final reduced program that produces the same prediction . • The final reduced program is 1-tree-minimal, which contains the key input features that the CI model must need for making the correct prediction .
After reducing a set of input programs of a target label, we extract the node type and token value from the abstract syntax tree (AST) of each reduced program. Every extracted element from reduced programs is considered as a candidate input feature. The most common elements are identified as label-specific key features and other uncommon elements are identified as input-specific sparse features.
Implementation. Our approach is model-agnostic and can be applied for various tasks and programming datasets. In this paper, for experimentation of our approach, we study two well-known code intelligence models (Code2Vec and Code2Seq), a popular code intelligence task (MethodName) and one commonly used programming language dataset (Java-Large) with different types of input programs.
3.0.1 Task. We use the method name prediction (MethodName [3]) task in this study. This task is commonly used by researchers in the code intelligence domain for various applications such as code summarization [3,6], representation learning [7,8], neural testing [13,16,25], feature extraction [17,24], and so on [4,21]. In the MethodName task, a model attempts to predict the name of a method from its body. For example, given the code snippet "void f(int a, int b) {int temp = a; a = b; b = temp;}", the Code2Vec model correctly predicts the method's name as "swap".
3.0.2 Models. We use the Code2Vec [8] and Code2Seq [7] code intelligence models for MethodName task. Both models use paths from abstract syntax trees (AST) to encode a program. Given a sample expression "a = b;", an example of path context in AST is "a, <NameExpr ↑ AssignExpr ↓ IntegerLiteralExpr>, b".
• Code2Vec. This model extracts a bag of path-contexts from the AST of the program where each pathcontext includes a pair of terminal nodes and the corresponding path between them. The model learns embeddings of these path-contexts during training and uses an attention mechanism to aggregate multiple path-contexts to a single code vector. The code vector is used as a representation of the program for making a prediction. • Code2Seq. This model also extracts a bag of path-contexts from the AST of the program but it sub-tokenizes each path-context. The Code2Seq model uses a bi-directional LSTM to encode paths node-by-node, and another LSTM to decode a target sequence one-by-one.
3.0.3 Dataset. For the MethodName task, we use the Java-Large dataset [7]. This dataset contains a total of 9, 500 Java projects from GitHub, where 9, 000 projects are for the training set, 200 projects for the validation set, and 300 projects for the test set. Using training set and validation set, we train both the Code2Vec and Code2Seq models.
3.0.4 Input Types. The dataset from GitHub is often imbalanced and contains different sizes and frequencies of input programs. Therefore, we choose different types of input programs from the Java-Large test set to evaluate the effectiveness of our approach in terms of reduction and feature extraction.
• Frequent Methods: We randomly sample a total of 100 input programs from the top-10 most occurring method names. • Rare Methods: We randomly sample a total of 100 input programs from the least occurring method names.
• Smaller Methods: We randomly sample a total of 100 input programs that contains less than 10 lines of code. • Larger Methods: We randomly sample a total of 50 input programs that have around 100 lines of code.
Moreover, to demonstrate the key input features, we select correctly predicted input programs from the ten most frequent labels of the Java-Large test set for feature extraction. Those labels (methods) are: equals, main, setUp, onCreate, toString, run, hashCode, init, execute, and get.
3.0.5 Syntax-unaware Reduction Technique. We use Sivand-DD [17] in this study which uses the Delta Debugging algorithm as the syntax-unaware program reduction technique. Zeller and Hildebrandt [26] have proposed the Delta Debugging algorithm to reduce the size of an input program. The algorithm iteratively splits an input program into multiple candidate programs by removing parts of the input program. The algorithm then checks if any resulting candidate program preserves the prediction of the model on the original input program. When the algorithm finds a candidate satisfying the property, it uses the candidate as the new base to be reduced further. Otherwise, the algorithm increases the granularity for splitting, until it determines that the input program cannot be reduced further. Considering the input reduction type (token or char), we use the following two variations of Sivand-DD: • Sivand-Token: In this token level reduction approach, Sivand-DD reduces the size of an input program token by token. We mostly use the Sivand-Token as the default baseline for Sivand-DD in this study. • Sivand-Char: In this char level reduction approach, Sivand-DD reduces the size of an input program char by char. We use the Sivand-Char approach to provide an additional explanation in Section 4.3 and Figure 3.
Rabin et al. [17] described in more detail how the Sivand-DD technique is applied in the workflow of reducing input programs for CI models.

Results
In this section, we present the result of our experiments on the Code2Vec and Code2Seq models and the Java-Large dataset for different input types.

Comparative Analysis
Here, we provide a systematic comparison between the syntax-guided program reduction technique and the syntax-unaware program reduction technique. In particular, we compare the syntax-guided Sivand-Perses and the syntax-unaware Sivand-DD in terms of token reduction, reduction steps and reduction time.

Token Reduction. The goal of Sivand-Perses and
Sivand-DD is to remove irrelevant tokens from an input program as much as possible while preserving the same prediction of the CI model. Figure 2a shows their such ability in reducing the size of the original input programs for different input types. We can see that, for all input types, Sivand-Perses reduces more tokens from input programs than Sivand-DD. On average, Sivand-Perses removes 20% more tokens from input programs than Sivand-DD. The difference is most significant (around 30%) in Large input types and less significant (around 5%) in Rare input types. This result suggests that Sivand-Perses is more powerful than Sivand-DD in reducing the size of an input program.

Reduction
Steps. The reduction is applied repeatedly to an input program until finding the final minimal program, from where no more tokens can be removed. From Figure 2b, we can see that Sivand-Perses on average can reach the final minimal program within 5 reduction steps. However, Sivand-DD makes around 20 reductions in Freqent-Rare-Small input types and more than 50 reductions in Large input type, to reach the final minimal program. The Sivand-DD reduces an input program by a sequence of tokens and backtracks if the reduced program is invalid, where Sivand-Perses can prune an entire sub-tree from the AST of the program and always generates a valid reduced program following grammar. Thus, Sivand-Perses takes a much lower number of reduction steps than Sivand-DD to reach the final minimal program.

Reduction Time.
We now compare the average time taken by Sivand-Perses and Sivand-DD for reducing an input program. As Sivand-DD takes excessive invalid steps, Sivand-Perses is expected to spend less time for program reduction. Figure 2c shows Sivand-Perses reduces an input program faster than Sivand-DD, for all input types and, especially so for Large input type. In Freqent-Rare-Small input types, both Sivand-Perses and Sivand-DD spend less than 2 minutes to reduce an input program and comparatively Sivand-Perses takes 30 seconds less time than Sivand-DD. In Large input types, Sivand-DD spends around 17 minutes for the reduction of a large program but Sivand-Perses takes only 8 minutes, which is around 50% less than Sivand-DD.
Observation 1: Sivand-Perses allows more token removal than Sivand-DD and always creates valid candidate programs. Compared to Sivand-DD, Sivand-Perses is more likely to reach the final minimal program in a smaller number of reduction steps, which decreases the total reduction time.

Key Input Features
Here, we provide the summary of extracted input features that CI models learn from training dataset for predicting  the target method name. In our experiment, we consider all tokens in reduced programs as candidate tokens for input features. We define a label-specific key feature as a candidate token that appears in at least 50% of the reduced programs, where other infrequent tokens are input-specific sparse features. For brevity and page limit, we only show the Top-10 most frequent methods in Table 1 and Table 2.
From Table 1, considering both Code2Vec and Code2Seq models, we can see that both Sivand-Perses and Sivand-DD identify around 50 tokens, in total, as label-specific key features in Top-10 methods. However, Sivand-DD contains a total of 324 candidate tokens in reduced programs, which is 1.36x times higher than Sivand-Perses that contains a total of 238 candidate tokens. In some methods, i.e., 'equals' and 'setUp', the total number of candidate tokens in Sivand-DD reduced programs is almost 2x times higher than the candidate tokens in Sivand-Perses reduced programs. This may suggest that the tokens found from the reduced programs of Sivand-DD are more input-specific while the tokens found from the reduced programs of Sivand-Perses are more labelspecific.
Furthermore, Table 2 shows the label-specific key features (sorted by their frequency of occurrences in reduced programs) extracted by Sivand-DD and Sivand-Perses from their reduced programs. These key input features from reduced programs can help to understand the prediction of the CI model for a target label. For example, Sivand-DD and Sivand-Perses reveal that "void, args, String, Exception" are key features for the 'main' method. It may highlight that a sample input program containing those tokens is more likely to be predicted as the 'main' method by CI models. However, the key feature of the Code2Seq model for the 'init' method is only "void" as it is the only common token in the reduced programs. Moreover, we did not find any common token in the reduced programs of the 'get' method for the Code2Seq model. Thus, by extracting key features from reduced programs of a target label, we Sivand-DD None Sivand-Perses None may also get an intuition about whether a CI model is learning a label correctly or not.
© Observation 2: According to our results, Sivand-Perses reveals more label-specific key features in its syntax-guided reduced programs, while Sivand-DD contains more input-specific sparse features in its syntax-unaware reduced programs. for a target label by a CI model (Table 2). Those different features can help us find multiple explanations for a specific prediction. For instance, Code2Seq predicts the code snippet in Figure 3a as the 'main' method. Sivand-DD with charbased program reduction (Sivand-Char) gives the minimal program in Figure 3b, that Code2Seq can predict as main. We can see the presence of the Main identifier in the method body of Figure 3b which is one of the key tokens for the target prediction. On the other hand, the Sivand-DD with token-based program reduction (Sivand-Token) gives the minimal program in Figure 3c, which suggests the argument args has an important role in the target prediction. However, with the AST-based program reduction (Sivand-Perses), the minimal program is shown in Figure 3d which highlights the signature of the method, for which Code2Seq still can predict the same target label. Having these multiple explanations can improve the transparency of the models' inferences. § ¦ ¤ ¥ Observation 3: Different program reduction techniques may provide additional explanations for a better understanding of a specific prediction.

Key Targeted Adversarial Attacks on Models
Here, we evaluate the importance of key input features in programs by evaluating the adversarial robustness [16,25] of CI models where we change the extracted key features. We generate adversarial examples by applying semanticpreserving variable renaming transformation on programs, similar to [16], where we separately change each variable and all of its occurrences in the program with token var. We particularly compare the prediction of CI models before and In key adversarial set, we also target the original input programs but generate candidate transformed programs by considering variables that occur in the key feature list.
In reduced adversarial set, we target the reduced input programs for generating candidate transformed programs. The results of change in prediction (misprediction) for variable renaming transformation are shown in Table 3. According to Table 3, on average, the number of generated candidate transformed programs in original adversarial set is around 3x times higher than the original programs, however, only around 12% of them trigger mispredictions. Next, the number of generated candidate transformed programs in key adversarial set is around 1.5x times higher than the original programs and trigger around 22% mispredictions. Although the key adversarial set contains almost 50% less candidate transformed programs than the original adversarial set, they trigger around 10% more mispredictions. On the other hand, the reduced programs are the minimal programs that CI models keep for preserving their target predictions. Therefore, the number of generated candidates transformed programs in reduced adversarial set is the lowest as there are fewer tokens to apply transformations. However, the transformation on reduced programs is more powerful and triggers the highest percentage of mispredictions, up to 65%. Moreover, comparing between Sivand-DD and Sivand-Perses, in most cases, Sivand-Perses generated candidates transformed programs show a higher rate of misprediction than Sivand-DD.

Observation 4:
The adversarial programs based on key input features trigger 10% more mispredictions with 50% fewer candidates. The Sivand-Perses generated candidate programs are more vulnerable to adversarial transformation than Sivand-DD, thus, highlighting the importance of key input features in programs.

Threats to Validity and Future Plan
Evaluation. We evaluated our approach for MethodName task with two CI models, four input types of randomly selected input programs, and Top-10 most frequent method names. Despite our best effort, it is possible that experiments with different models, tasks, and datasets may produce different results. Our further plan includes a detailed study with a variety of models, tasks, and larger datasets. Challenges. One challenge to integrating Perses is that it loads the model in each reduction step while Delta Debugging loads the model once at the beginning of reduction. For a fair comparison between them, we only consider the program reduction time and ignore the model loading time. We are working on optimizing the model loading time for Perses. Another challenge to integrating Delta Debugging, when there are multiple subsets that hold the same target criteria, is that Delta Debugging sometimes gets stuck at that point. To keep the reduction process working, we temporarily used a timer to stop the current step and jump to the next step.

Conclusion
In this paper, we apply the syntax-guided program reduction technique, Sivand-Perses, for reducing an input program while preserving the same prediction of the CI model. The goal is to improve the transparency of models by extracting key input features from reduced programs. We evaluate Sivand-Perses on two popular CI models across four types of input programs for the method name prediction task. Our results suggest that the syntax-guided program reduction technique (Sivand-Perses) significantly outperforms the syntax-unaware program reduction technique (Sivand-DD) in reducing different input programs. Moreover, we extract key input features that CI models learn for a target label, by reducing some input programs of that label. The result shows that Sivand-Perses keeps more label-specific key input features in its syntax-guided reduced programs than in Sivand-DD's syntax-unaware reduced programs. We also observe that different program reduction techniques may provide additional explanations for a better understanding of a specific prediction.

Broader Impact
In this section, we discuss the potential misuses and negative impacts of our approach. Our work focuses on improving the transparency of neural code intelligence models by reducing input programs with a syntax-guided technique and extracting key features that models learn from input programs. Therefore, in the negative scenario, attackers can try to force a model to make mispredictions by applying the key targeted adversarial attack (section 4.4), thus may hamper the wider adoption of the model. However, our work also highlights multiple explanations for a prediction (section 4.3), which can motivate researchers to interpret and improve the robustness of a model against those malicious activities.