{'fraction_non_alphanumeric': 0.03672663732850116, 'fraction_numerical': 0.011648977478643541, 'mean_word_length': 4.703081749400258, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Evasion and Counter-Evasion Study in Malicious Websites Detection

Malicious websites are a major cyber attack vector, and effective detection of them is an important cyber defense task. The main defense paradigm in this regard is that the defender uses some kind of machine learning algorithms to train a detection model, which is then used to classify websites in question. Unlike other settings, the following issue is inherent to the problem of malicious websites detection: the attacker essentially has access to the same data that the defender uses to train its detection models. This 'symmetry' can be exploited by the attacker, at least in principle, to evade the defender's detection models. In this paper, we present a framework for characterizing the evasion and counter-evasion interactions between the attacker and the defender, where the attacker attempts to evade the defender's detection models by taking advantage of this symmetry. Within this framework, we show that an adaptive attacker can make malicious websites evade powerful detection models, but proactive training can be an effective counter-evasion defense mechanism. The framework is geared toward the popular detection model of decision tree, but can be adapted to accommodate other classifiers.


I. INTRODUCTION
Compromising websites and abusing them to launch further attacks (e.g., drive-by-download [7], [20]) have become one of the mainstream attack vectors. Unfortunately, it is infeasible, if not impossible, to completely eliminate such attacks, meaning that we must have competent solutions that can detect compromised/malicious websites as soon as possible. The dynamic approach, which is often based on client honeypots or variants, can detect malicious websites with high accuracy, but is limited in terms of its scalability. The static approach, which often analyzes the website contents and then uses some detection models (e.g., decision trees) to classify them into benign/malicious classes, is very efficient, but suffers from its limited success in dealing with sophisticated attacks (e.g. JavaScript obfuscation). This hints that there is perhaps some inherent limitation in the trade-off between scalability and detection effectiveness.
In this paper, we bring up another dimension of the problem, which may have a fundamental impact on the aforementioned inherent limitation. Unlike in other settings, the following issue is inherent to the problem of malicious websites detection: The attacker essentially has access to the same data that the defender uses to train its detection models. This 'symmetry' could be exploited by the attacker to evade the defender's detection models. This is because the attacker can effectively train and obtain (almost) the same detection models, and then exploit them to make other malicious websites evasive. This is possible because the attacker can manipulate the contents of malicious websites during the course of compromising them, or after they are compromised but before they are analyzed by the defender's detection models. This is feasible because the attacker controls the malicious websites.
More specifically, we make two contributions. First, we propose a framework for characterizing the evasion and counterevasion interactions between the attacker and the defender. The framework accommodates a set of adaptive attacks against a class of detection models known as decision trees [22], which have been widely used in this problem domain. The framework also accommodates the novel idea of proactive training as the counter-evasion mechanism against the adaptive attacks, where the defender proactively trains its detection models while taking adaptive attacks into consideration. Although the framework is geared towards decision trees, it can be adapted to accommodate other kinds of classifiers.
Second, we use a dataset that was collected during the span of 40 days to evaluate the evasion power of adaptive attacks and the counter-evasion effectiveness of proactive training. Experimental results show that an adaptive attacker can make malicious websites evade powerful detection models, but proactive training can be an effective counter-evasion defense mechanism. In order to deepen our understanding of the evasion and counter-evasion interactions, we also investigate which features (or attributes) of websites have a high security significance, namely that their manipulation causes the misclassification of malicious websites. Surprisingly, we find that the features of high security significance are almost different from the features that would be selected by the standard feature selection algorithms. This suggests that we might need to design new machine learning algorithms to best fit the domain of security problems. Moreover, we find that the detection accuracy of proactively-trained detection models increases with the degree of the defender's proactiveness (i.e. the number of training iterations). Finally, we find that if the defender does not know the attacker's adaptation strategy, the defender should adopt the full adaptation strategy that will be described later.
The rest of the paper is organized as follows. Section II briefly reviews the context of the present study. Section III investigates the framework of evasion and counter-evasion interactions. Section IV evaluates the effectiveness of the framework. Section V discusses related prior work. Section VI concludes the paper.

II. PRELIMINARIES
In order to illustrate the power of adaptive attacks and the effectiveness of our counter-measure against them, we need to consider some concrete detection scheme. Since J48 classifiers are known to be successful in detecting malicious websites [3], [29], [30], [15], [6], [14], we adopt the detection scheme we proposed in [30] as the starting point of the present study. We showed in [30] that J48 classifier outperforms Naive Bayes, Logistic and SVM classifiers.
We also inherit the data collection method described in [30]. At a high level, a crawler is used to fetch the website content corresponding to an input URL, benign and malicious alike. Each URL is described by 105 application-layer features and 19 network-layer features [30]. We now briefly review the following 16 features that will be encountered later: URL_length (length of URL); Content_length (the content-length field in HTTP header, which may be manipulated by malicious websites); #Redirect (number of redirects); #Scripts (number of scripts); #Embedded_URL (number of URLs embedded); #Special_character (number of special characters in a URL); #Iframe (number of iframes); #JS_function (number of JavaScript functions in a website); #Long_string (number of strings with 51 or more letters in embedded JavaScript programs); #Src_app_bytes (number of bytes communicated from crawler to website); #Local_app_packet (number of crawler-to-website IP packets, including redirects and DNS queries); Dest_app_bytes (volume of website-to-crawler communications); Duration (the time it takes for the crawler to fetch the contents of a website, including rediects); #Dist_remote_tcp_port and #Dist_remote_IP (number of distinct TCP ports and IP addresses the crawler uses to fetch websites contents, respectively); #DNS_query (number of DNS queries); #DNS_answer (number of DNS server's responses).
The main notations are summarized as follows. ST, C, F adaptation strategy ST, manipulation algorithm F, manipulation constraints C s R ← S assigning s as a random member of set S III. EVASION AND COUNTER-EVASION FRAMEWORK Adaptive attacks are possible because an attacker can collect the same data as what is used by the defender to train a detection scheme. The attacker also knows the machine learning algorithm(s) the defender uses or even the defender's detection scheme. To accommodate the worst-case scenario, we assume there is a single attacker that coordinates the compromise of websites (possibly by many sub-attackers). This means that the attacker knows which websites are malicious, while the defender aims to detect them. In order to evade detection, the attacker can manipulate some features of the malicious websites. The manipulation operations can take place during the course of compromising websites, or after compromising websites but before they are classified by the defender's detection scheme.

A. Framework Overview
We describe adaptive attacks and countermeasures in a modular fashion, by using eight algorithms whose caller-callee relation is highlighted in Figure 1. Algorithm 1 is the attacker's main algorithm, which calls Algorithm 2 for preprocessing, and calls Algorithm 4 or Algorithm 5 for selecting features to manipulate and for determining the manipulated values for the selected features. Both Algorithm 4 and Algorithm 5 call Algorithm 3 to compute the escape intervals for the features that are to be manipulated. An escape interval defines the interval from which the manipulated value of a feature should be taken so as to evade detection. Algorithm 6 is the defender's main algorithm, which calls Algorithm 7 for proactive training of detection schemes. For this purpose, the defender can have access to its own proactive manipulation algorithm. In our experiments, we let the defender have access to the manipulations algorithms that are available to the attacker, namely Algorithm 4 and Algorithm 5. This is sufficient for the purpose of understanding the effectiveness of proactive training and detection against adaptive attacks under various algorithm/parameter possibilities, such as: the defender correctly or incorrectly "guess" the manipulation algorithm or parameters that are used by the attacker, and the relatively more effective proactive training strategy against a class of adaptive attacks. In order to evaluate the effectiveness of proactive training and detection against adaptive attacks, we use an "artificial" Algorithm 8, which is often implicit in most real-life defense operations.

B. Evasion Model and Algorithms
In our model, a website is represented by a feature vector. We call the feature vector representing a benign website benign feature vector, and the feature vector representing a malicious website malicious feature vector. Denote by D ′ 0 the defender's training data, namely a set of feature vectors corresponding to a set of benign websites (denoted by D ′ 0 .benign) and malicious websites (denoted by D ′ 0 .malicious). The defender uses a machine learning algorithm MLA to learn a detection scheme M 0 from D ′ 0 (i.e., M 0 is learned from one portion of D ′ 0 and tested via the other portion of D ′ 0 ). As mentioned above, the attacker is given M 0 to accommodate the worstcase scenario. Denote by D 0 the set of feature vectors that are to be classified by M 0 to determine which feature vectors (i.e., the corresponding websites) are malicious. The attacker's objective is to manipulate the malicious feature vectors in D 0 into some D α so that M 0 (D α ) has a high false-negative rate, where α > 0 represents the number of iterations (or rounds) the attacker conducts the manipulation operations. Algorithm 1 describes the adaptive attack. As highlighted in Figure 2, we consider three basic adaptation strategies.
. . , α, and F is a randomized manipulation algorithm, meaning that where detection schemes M 1 , . . . , M α are respectively learned from D 1 , . . . , D α using the defender's machine learning algorithm MLA (also known to the attacker). • ST == full-adaptation: The attacker sets the ma- . .) is a preprocessing algorithm for "aggregating" sets of feature vectors D 0 , D 1 , . . . into a single set of feature vectors, F is a manipulation algorithm, M 1 , . . . , M α are learned respectively from D 1 , . . . , D α by the attacker. Algorithm 2 is a concrete preprocessing algorithm. Its basic idea is the following: since each malicious website corresponds to m malicious feature vectors that respectively belong to D 0 , . . . , D m−1 , the preprocessing algorithm randomly picks one of the m malicious feature vectors to represent the malicious website in D. It is worth mentioning that one can derive some hybrid attack strategies from the above three basic strategies. We also note that the attack strategies and the manipulation constraints are independent of the detection schemes, but the manipulation algorithms would be specific to the detection schemes. D ← D ∪ D0.benign 6: end for 7: return D Manipulation Constraints. For a feature X whose value is to be manipulated, the attacker needs to compute X.escape interval, which is a subset of feature X's domain domain(X) and can possibly cause the malicious feature vector to evade detection. Feature X's manipulated value is randomly chosen from its escapte interval, which is calculated using Algorithm 3, while taking as input X's domain constraints and semantics constraints.
INPUT: X is feature for manipulation, M is detection scheme, C represents constraints OUTPUT: X's escape interval 1: domain constraint ← C.domain map(X) 2: semantics constraint ← C.semantics map(X) {∅ if X cannot be manipulate due to semantics constraints} 3: escape interval ← domain constraint ∩ semantics constraint 4: return escape interval Algorithm 3 is called because the manipulation algorithm needs to compute the interval from which a feature's manipulated value should be taken. Specifically, the constraints are the following.
• Domain constraints: Each feature has its own domain of possible values. This means that the new value of a feature after manipulation must fall into the domain of the feature. Let C.domain map be a table of (key, value)   Manipulation Algorithms. As mentioned in Section II, we adopt the J48 classifier detection scheme, where a J48 classifier is trained by concatenating the application-and network-layer features corresponding to the same URL [30]. We present two manipulation algorithms, called F 1 and F 2 , which exploit the defender's J48 classifier to guide the manipulation of features. Both algorithms neither manipulate the benign feature vectors (which are not controlled by the attacker), nor manipulate the malicious feature vectors that are already classified as benign by the defender's detection scheme (i.e., false-negative). Both algorithms may fail, while brute-forcing may fail as well because of the manipulation constraints.
Since the manipulation algorithms are inevitably complicated, in the following we will present their basic ideas and sketched algorithms. The notations used in the algorithms are: for node v in the classifier, v.f eature is the feature associated to node v, and v.value is v.f eature's "branching" value as specified by the classifier (a binary tree with all features numericalized).
Manipulation Algorithm F 1 is described as Algorithm 4. The basic idea underlying this manipulation algorithm is the following: for every malicious feature vector in D, there is a unique path (in the J48 classifier M ) that leads to a malicious leaf, which indicates that the feature vector is malicious. We call the path leading to malicious leaf a malicious path, and the path leading to a benign leaf (which indicates a feature vector as benign) a benign path.
By examining the path from the malicious leaf to the root, say malicious leaf → v 2 → . . . → root, and identifying the first inner node, namely v 2 , the algorithm attempts to manipulate fv.(v 2 .f eature).value so that the classification can lead to malicious leaf 's sibling, say v 2,another child , which is guaranteed to exist (otherwise, v 2 cannot be an inner node). Note that there must be a sub-path rooted at v 2,another child that leads to a benign leaf (otherwise, v 2 cannot be an inner node as well), and that manipulation of values of the features corresponding to the nodes on the sub-tree rooted at while v is not benign leaf do 5: if v is an inner node then 6: v ← v.Child based on decision tree rule 7: update corresponding feature interval 8: else if v is a malicious leaf then 9: compute the corresponding feature's escape interval by calling Algorithm 3 10: pick a value z in the escape interval uniformly at random 11: set the corresponding feature's value to z 12: v ← v.sibling 13: end if 14: end while 15: end for 16: return manipulated feature vectors D v 2,another child will preserve the postfix v 2 → . . . → root.
To help understand the manipulation algorithm, let us look at one example. At a high-level, the attacker runs AA("J48", M 0 , D 0 , ST, C, F 1 , α = 1) and therefore which ends at malicious leaf v 1 . The manipulation algorithm first identifies malicious leaf v 1 's parent node v 9 , and manipulates X 9 's value to fit into v 1 's sibling (v 8 ). Note that X 9 's escape interval is as: where Domain(X 9 ) = [min 9 , max 9 ], [min 9 , 7] corresponds to node v 9 on the path, and [min 0 , 13] corresponds to node v 0 on the path. The algorithm manipulates X 9 's value to be a random element from X 9 's escapte interval, say 8 ∈ (7, 13], which causes the manipulated feature vector to evade detection because of decision path: and ends at benign leaf v 3 .
Manipulation Algorithm F 2 is described as Algorithm 5. The basic idea underlying this manipulation algorithm is to first extract all benign paths. For each feature vector fv ∈ D.malicious, F 2 keeps track of the mismatches between fv and all benign paths. The algorithm attempts to manipulate as few "mismatched" features as possible to evade M .

Algorithm 5 Manipulation algorithm F 2 (M, D, C)
INPUT: J48 classifier M , feature vector set D(malicious ∪ benign), manipulation constraints C OUTPUT: manipulated feature vectors 1: create interval vector for features along every benign path and store in Paths. 2: for all feature vector ∈ D.malicious do 3: for all P ath ∈ Paths do 4: compare feature vector to P ath and record the mismatch feature number 5: end for 6: sort Paths in ascending order of mismatch feature number 7: for all P ath ∈ Paths until successfully manipulated do 8: for all mismatch feature ∈ Paths do 9: get escape interval by calling Algorithm 3 10: pick a value n in escape interval at random 11: set feature vector's corresponding feature value to n To evade detection, the attacker can compare the feature vector to the matrix of two benign paths. For the benign path v 3 → v 7 → v 8 → v 9 → v 10 → v 0 , the feature vector has three mismatches, namely features X 4 , X 9 , X 18 . For the benign path v 13 → v 11 → v 12 → v 0 , the feature vector has two mismatches, namely X 9 and X 1 . The algorithm first processes the benign path ending at node v 13 . The algorithm will try to manipulate X 9 and X 1 's values to reach the benign leaf. Suppose on the other hand, that X 10 cannot be manipulated without violating the constraints. The algorithm stops with this benign path and considers the benign path end at node v 3 . If the algorithm fails with this benign path again, the algorithm will not manipulate the feature vector and leave it to be classified as malicious.

C. Counter-Evasion Algorithms
We have showed that adaptive attacks can ruin the defender's (non-proactive) detection schemes. Now we investigate countermeasure against adaptive attacks. The counter-measure is based on the idea of proactive training & detection. Algorithm 6 describes the proactive detection algorithm. The basic idea of this algorithm is to call the proactive training algorithm to generate a set of proactively trained detection schemes, denoted by M † 1 , . . . , M † γ . These detection schemes are derived from, among other things, M 0 , which is learned from D ′ 0 using MLA. It is important to note that D ′ 0 = D ′ 0 .benign ∪ D ′ 0 .malicious, 0 .malicious, Dα (α unknown to defender) is set of feature vectors for classification (where the malicious websites may have been manipulated by the attacker), STD is defender's adaptation strategy, FD is defender's manipulation algorithm, C is set of constraints, γ is defender's number of adaptations rounds OUTPUT: malicious vectors fv ∈ Dα where D ′ 0 .benign is a set of websites that are known to be benign (ground truth) and cannot be manipulated by the attacker, D ′ 0 .malicious is a set of websites that are known to be malicious (ground truth). A website is classified as malicious if the non-proactive detection scheme M 0 classifies it as malicious, or at least ⌊(γ+1)/2⌋+1 of the proactively trained detection schemes classify it as malicious. This is meditated to accommodate that the defender does not know a priori whether the attacker is adaptive or not, When the attacker is not adaptive, if STD == parallel-adaptation then 5: else if STD == sequential-adaptation then 7: else if STD == full-adaptation then 9: 10: Essentially, the evaluation algorithm calls the defender's protection detection to generate a set of proactively trained detection schemes, and calls the attacker's adaptive attack algorithm to manipulate the malicious websites (i.e., selecting some of their features and manipulating their values to evade a given detection scheme). By varying the adaptation strategies and the parameters, we can evaluate the effectiveness of proactive training & detection against the adaptive attacks. The parameter space of the evaluation algorithm includes at least 108 scenarios: the basic adaptation strategy space ST A × ST D is 3×3 (i.e., not counting any hybrids of parallel-adaptation, sequential-adaptation and full-adapatation), the manipulation algorithm space F A × F B is 2 × 2, and the adaptation round parameter space is at least 3 (α >, =, < γ).

IV. EVALUATING EFFECTIVENESS OF THE COUNTER-EVASION ALGORITHMS
We use the standard metrics, including false-negative and false-positive rates [30], to evaluate the effectiveness of counterevasion algorithms.

A. Data Description
The dataset used in this paper consists of a 40day URLs. Malicious URLs are downloaded from blacklists: compuweb.com/url-domain-bl.txt, malware.com.br, malwaredomainlist.com, zeustracker.abuse.ch and spyeyetracker.abuse.ch and further confirmed with the high-interaction client honeypot Capture-HPC [24]. Benign URLs are obtained from alexa.com, which lists the top 10,000 websites that are supposed to be well protected. The test of blacklist URLs using high-interaction client honeypot confirmed our observation that some or many blacklist URLs are not accessible any more and thus should not be counted as malicious URLs.
The daily average number of malicious websites listed in blacklists is 6763 and the daily average number of malicious websites after being verified by Capture-HPC is 838. The total number of distinct malicious websites we found in 40 days are 17091. The daily average number of benign websites is 10,000. By eliminating non-accessible benign websites, the daily average number of benign websites is 9501 in 40 days. According to [26] and our experiment results, it can achieve best detection rate, when the rate between the number of benign websites and malicious websites are 4:1. So we choose all malicious websites and 4 times of benign websites as our training and testing data. Table I summarizes the results of adaptive attack AA("J48", M 0 , D 0 , ST, C, F, α = 1) based on the 40-day dataset mentioned above. The experiment can be more succinctly represented as M 0 (D 1 ), meaning that the defender is static (or non-proactive) and the attacker is adaptive with α = 1, where D 1 is the manipulated version of D 0 . Note that in the case of α = 1, the three adaptation strategies lead to the same D 1 as shown in Figure 2. From Table I  Having observed the phenomenon that manipulation of some features' values can essentially make the detection schemes useless, it would be natural to ask which features are often manipulated for evasion? To look into the question, we notice that many features are manipulated over the 40 days, but only a few are manipulated often. F 1 most often (i.e., > 150 times each day for over the 40 days) manipulates three application-layer features -URL_length, Content_length, #Embedded_URLs -and two network-layer features -Duration and #Local_app_packet. On the other hand, F 2 most often (i.e., > 150 times) manipulates two application-layer features -#Special_characters and Content_lengthand one network-layer feature -Duration.

B. Effectiveness of the Evasion Attacks
The above discrepancy between the frequencies that features are manipulated can be attributed to the design of the manipulation algorithms. Specifically, F 1 seeks to manipulate features that are associated to nodes that are close to the leaves. In contrast, F 2 emphasizes on the mismatches between a malicious feature vector and an entire benign path, which represents a kind of global search and also explains why F 2 manipulates fewer features.
We also want to know why these features have such high security/evasion significance? The issue is important because identifying the "important" features could lead to deeper insights. We compare the manipulated features to the features that would be selected by a feature selection algorithm for the purpose of training classifiers. To be specific, we use the InfoGain feature selection algorithm because it ranks the contributions of individual features [30]. We find that among the manipulated features, URL_length is the only feature among the five InfoGain-selected application-layer features, and #Dist_remote_TCP_port is the only feature among the four InfoGain-selected network-layer features. This suggests that the feature selection algorithm does not necessarily offer good insights into the importance of features from a security perspective.
The standard feature-selection algorithms are almost useless to find indicative features from the perspective of evading classifiers. There is still gap between our results and the "optimal" solutions based on security semantics; it's an open problem to bridge the gap, because classifiers are "black-box" that don't really accommodate "security semantics" of features. Table II summarizes the effectiveness of proactive defense against adaptive attacks. We make the following observations. First, if the defender is proactive (i.e., γ > 0) but the attacker is non-adaptive (i.e., α = 0), the false-negative rate drops from 0.79% in the baseline case to some number belonging to interval [0.23%, 0.56%]. The price is: the detection accuracy drops from 99.68% in the baseline case to some number belonging to interval [99.23%, 99.68%] the false-positive rate increases from 0.14% in the baseline case to some number belonging to interval [0.20%, 0.93%], The above observations suggest: the defender can always use proactive detection without worrying about side-effects (e.g., when the attacker is not adaptive). This is because the proactive detection algorithm PD uses M 0 (D 0 ) as the first line of detection.

C. Effectiveness of the Counter-Evasion Algorithms
Second, when ST A = ST D = 0, it has a significant impact whether or not they use the same manipulation algorithm. This phenomenon also can be explained by that the features that are often manipulated by F 1 are very different from the features that are often manipulated by F 2 . More specifically, when F A = F D , the proactively learned classifiers M † 1 , . . . , M † γ would capture the "maliciousness" information embedded in the manipulated data D α ; this would not be true when F A = F D . Moreover, the sequential adaptation strategy appears to be more "oblivious" than the other two strategies in the sense that D α preserves less information about D 0 . what adaptation strategy should the defender use to counter ST A = sequential? Table III shows that the attacker does not have an obviously more effective counter full adaptation strategy. This hints that the full strategy may be a kind of equilibrium strategy because both attacker and defender have no significant gains by deviating from it. This inspires an important problem for future research: Is the full adaptation strategy (or variant of it) an equilibrium strategy?
Third, Table II shows that when ST D = ST A , the attacker can benefit by increasing its adaptiveness α. Table III exhibits 9], and that the detection accuracy is averaged over the 40 days. We observe that the detection accuracy in most cases there is a significant increase in detection accuracy when the defender's proactiveness matches the attacker's adaptiveness.
the same phenomenon when ST D = ST A . In order to see the impact of defender's proactiveness as reflected by γ against the defender's adaptiveness as reflected by α, we plot in Figure 4 how the detection accuracy with respect to (γ − α) under the condition F D = F A and under various ST D × ST A combinations. We observe that roughly speaking, as γ increases to exceed the attacker's adaptiveness α (i.e., γ changes from γ < α to γ = α to γ > α), the detection accuracy may have a significant increase at γ − α = 0. Moreover, we observe that when ST D = full, γ − α has no significant impact on the detection accuracy. This suggest that the defender should always use the full adaptation strategy to alleviate the uncertainty about the attacker's adaptiveness α.
The evasion attack is closely related to the problem of adversarial machine learning, where the attacker aims to evade an detection scheme that is derived from some machine learning method [1], [27]. Perdisci et al. [19] investigated how to make the detection harder to evade. Nelson et al. [18] assumed the attacker has black-box access to the detection mechanism. Dalvi et al. [8] used Game Theoretic method to study this problem in the setting of spam detection by assuming the attacker has access to the detection mechanism. Our model actually gives attacker more freedom because the attacker knows the data defender collected.

VI. CONCLUSION AND FUTURE WORK
We formulated a model of adaptive attacks by which the attacker can manipulate the malicious websites to evade detection. We also formulated a model of proactive defense against the adaptive attacks. Experimental results based on a 40-day dataset showed that adaptive attacks can evade non-proactive defense, but can be effectively countered by proactive defense.
In the full version of the present paper, we will address correlation constraints between features, which is omitted due to space limitation. This study also introduces a set of interesting research problems, including: Are the same kinds of results/insights applicable to classifiers other than decision trees? Is the full adaptation strategy indeed a kind of equilibrium strategy? What is the optimal manipulation algorithm (if exists)? How can we precisely characterize the evadability caused by adaptive attacks in this context? What is the optimal time resolution at which the defender should proactively train its detection schemes (e.g., hour or day)?