{'fraction_non_alphanumeric': 0.03218152157793267, 'fraction_numerical': 0.010084532107370607, 'mean_word_length': 4.959787892178524, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review.

Cloud computing has become a powerful and indispensable technology for complex, high performance and scalable computation. The exponential expansion in the deployment of cloud technology has produced a massive amount of data from a variety of applications, resources and platforms. In turn, the rapid rate and volume of data creation has begun to pose significant challenges for data management and security. The design and deployment of intrusion detection systems (IDS) in the big data setting has, therefore, become a topic of importance. In this paper, we conduct a systematic literature review (SLR) of data mining techniques (DMT) used in IDS-based solutions through the period 2013-2018. We employed criterion-based, purposive sampling identifying 32 articles, which constitute the primary source of the present survey. After a careful investigation of these articles, we identified 17 separate DMTs deployed in an IDS context. This paper also presents the merits and disadvantages of the various works of current research that implemented DMTs and distributed streaming frameworks (DSF) to detect and/or prevent malicious attacks in a big data environment.


I. INTRODUCTION
Enterprises, like government, finance, industry, and health care, need strong, reliable cybersecurity solutions [1]. Undoubtedly, the challenge has increased dramatically with the big data era from the current implemented solutions, such as traditional intrusion detection systems (IDS) [2]. The substantial recent growth of technological capabilities paves the way to cyber threats that target individuals or enterprises with various malicious attacks. In this threat environment, traditional security protections like firewalls, anti-virus software, and virtual private networks (VPNs) are not always sufficient, and robust mechanisms for detecting intrusions must also be deployed. For that reason, deploying an IDS along with the other traditional security systems is an essential strategy [3]. An IDS is automated software used to analyze the network traffic to detect and/or prevent malicious attacks [4]. It can be classified by two common methods namely, misuse-based detection (MD) and anomaly-based detection (AD). In the first approach, classification of the attack is based specifically on known patterns, called signatures [5]. AD, on the other side, seeks to detect abnormal patterns or behavior, with the key challenge of distinguishing between normal and abnormal patterns with low error [5]. There are different studies discussing the challenges facing IDS especially with the rise of big data era, which brings up different types of malicious attacks. Accordingly, special tools are required to accommodate the scalability of big data to detect cyber threats. The notion of big data is an umbrella term that encompasses the recent trends toward collection, storage and analysis of enormous volumes of data. Popular big data management tools include Hadoop, Spark, and Shark.
To the best of our knowledge, no prior systematic literature review addresses research questions (RQs) about implemented DMTs in the context of IDS, which has motived this work. In related work, Ashraf and Habaebi [6] discussed the threat approaches in the Internet of Things using an autonomic taxonomy. Ali et al. [7] summarized the security challenges and vulnerabilities in the mobile cloud computing. Ahmed et al. [8] presented the major categories of anomaly detection including statistical, classification, and clustering. They also discussed the dataset challenges of network IDS. Zuech et al. [9] reviewed the problem of big heterogeneous data. Injadat et al. [10] summarized the data mining techniques implemented in social media. Alguliyev and Imamverdiyev [11] analyzed the big data applications related to information security challenges and illustrated the research directions at the same domain. In this study, we conduct a systematic literature review (SLR) following the framework of Kitchenham and Charters, [12] exploring research into DMTs in the big data setting for IDS. The study covers the period between Jan 2013 and Dec 2018, evaluates, and analyzes research conducted in this space. This paper is divided into four sections. Section II describes our review methodology. Section III presents the results and discussion. Section IV includes the conclusion and recommendations for future work.

II. METHODOLOGY
The systematic methodology of this review consists of three stages [12]: planning, conducting, and reporting, with each stage consisting of several sub-stages. Detailed information of this methodology is presented in the following subsections. Fig. 1 demonstrates this methodology.

A. Research Questions (RQs)
The key element of this study is to analyze and report the empirical research about the DMT with big data in the

B. Study Selection
This research requires comparative investigation across the relevant articles. The digital libraries used in our review included: IEEE Xplore, ACM, Web of Science, Elsevier, Springer, and Wiley. Our initial search returned 545 articles. Tangential and duplicate articles were then identified during a subsequent filtration step. Next, we applied our inclusion and exclusion criteria to exclude the irrelevant articles. Next, we applied our Quality Assessment Rules (QARs) as a benchmark to evaluate the quality of the candidate articles. The inclusion criteria consist of the following rules: 1) Include articles that use data mining techniques with big data in IDS. 2) Include comparative studies of data mining techniques with big data in IDS. 3) Include the most recent version of articles (if any). 4) Consider the articles during the publication period between Jan 2013 and Dec 2018. On the other hand, exclusion criteria consisted of these rules: 1) Exclude articles that involve data mining with big data but are not related to IDS. 2) Exclude articles that involve IDS but are not related to data mining with big data. 3) Exclude articles that involve data mining in IDS but are not related to big data. 4) Exclude articles not categorized as peer-reviewed journal or conference papers. After applying the inclusion and exclusion criteria, we obtained 53 journal and conference articles as the primary source of the quality assessment stage.

C. Quality Assessment Rules (QAR)
As a step to guarantee the quality of the articles, 10 quality assessment questions were identified as follows: QAR1: Is the research purpose/problem clearly defined? QAR2: Are the DMT utilized with big data clearly defined? QAR3: Is the IDS principle clearly described? QAR4: Is the data preprocessing in the experiment well suited for the intended research? QAR5: Is the employed dataset clearly defined? QAR6: Are the big data mining tools clearly pinpointed? QAR7: Are the merits and shortcomings of the employed big data analysis tool, software, or algorithm addressed? QAR8: Is the background of attacks clearly depicted? QAR9: Does the research apply accurate evaluation metrics? and QAR10: Are the experiment results and findings are clearly reported?
Based on these QARs, we evaluated the 53 articles to assure the richness and suitability of the final outcome. Each question was graded from 0 to 1 mark, where each QAR was scored as follows: "Excellent" = 1, "Very good" = 0.75, "Good" = 0.5, "Poor" = 0.25, "not answered" = 0. Accordingly, article with score 5 or more was considered for the data extraction stage. Overall, a total of 32 articles were considered in the data extraction stage. Table I list the selected articles group by the final score.

D. Extract and Synthesize Data
The aim of this stage is to extract information that will answer the RQs. For the sake of quality assurance, each author plays the role of extractor and checker of others? work.
To ensure the consistency of the analytical results, common names were used for the same DMT, software, or big-data tool. For instance, Bayesian Network replaced the other related names such as Naive Bayes, Naïve Bayes, or Naïve Bayesian. Different methods were adopted from [12] to provide evidence of the collected data that are sought to provide answers to the present RQs. We used narrative synthesis (RQ1, RQ3, and RQ4), binary outcomes (All RQs), and reciprocal translation (RQ2) methods since those methods outfit the answers of our RQs. Table II shows the RQs that were answered by each selected article.

III. RESULTS AND DISCUSSION
A. RQ1: Which data mining techniques with big data have been used in IDS? Next, we enumerated the DMTs used in the feature selection/extraction (FS) phase since it is a vital step in making classification computationally feasible and reasonable accurate.  Table III shows the pre-processing techniques applied by the included articles.
It is worth mentioning, as depicted in Table III, that a big number of experiments did not consider the FS step in their approach, although it is important step to enhance the speed and effectiveness of the proposed models, especially in the big data setting. Section III-B includes further discussion about the utilized DMTs. The aim of this RQ is to enumerate the most popular evaluation metrics utilized by the respective IDSs. Overall, 19 approaches were identified in the related literature, detailed explanation about the techniques are discussed in [45]. We categorized our findings into two metrics: efficiency and performance. The efficiency metric includes 16 methods: Accuracy  Fig. 4 shows that TPR, Acc, and FPR are the most commonly used for efficiency measurement, while the popular performance measurements are TST, CC, and TRT. Even though evaluation metrics are considered vital to measure and evaluate the performance of IDSs, some papers did not validated their results sufficiently insofar as they relied on a single evaluation metric as a performance indicator, which is known to generate misleading results in some cases [46]. For instance, P2, P9, P11, and P13 reported either Acc, ER, or FPR in their studies. In the same context, we observed 12 articles did not consider the performance metrics utilization at all. Attacks can be classified into four major types: Denial of Service (DoS), Remote to User (R2L), User to Root (U2R), and Probing. Based on the data extracted, we classified the results either based on realistic datasets captured from blogs, profiles, and network traffic or benchmark datasets including DARPA, KDD Cup 99, NSL-KDD, ISCX-UNB, UNSW-NB15, and CAIDA. Detailed information about these attacks and datasets are discussed in [47]. We noticed, as shown in Fig. 5, the majority of the experiments are designed to provide binary classification of behaviors, for example, as being either normal or malicious. Furthermore, none of the real-time IDS were able to classify intrusions into the four major types. It is worth to mention that a significant number of the experiments are based the DARPA/KDD datasets which they considered obsolete and does not capture contemporary attack types.

E. Further discussion
The discussion in this section pinpoints the contributions and future works of the relevant studies in this SLR. The IV. CONCLUSIONS This research explored the relevant empirical studies published in journals and conferences to investigate the utilized DMTs with big data in IDS domain. We manually examined an initial 545 papers, which we eventually reduced to include 32 relevant papers. The findings of this work provide an integrated and unified view of the extracted data to answer the proposed RQs. The conclusion of this SLR is summarized as follows: • RQ1: The most commonly used DMT with big data in IDs are: DT, BN, k-Means, ANN, and SVM. Likewise, PCA was the most used technique in FS. Furthermore, a large fraction of papers did not consider FS in their approach, which considered an essential step to improve the overall performance. • RQ2: Hadoop was the most popular DSF with a frequency of 12 experiments. However, other big data tools, such as Spark reported better performance. We also provide an additive summary that highlights strengths and weaknesses of the used big data tools, software, or mining algorithms in the field of IDS. • RQ3: We enumerated the evaluation metrics in terms of efficiency and performance. The most used efficiency metrics are TPR, Acc, and FPR. On the other hand, TST, CC, and TRT were reported more frequently as a performance metrics. In addition, we observed inconsistencies that exist in the evaluation metrics used in IDS. • RQ4: We noticed that most of the experiments have classified attacks as either normal or malicious. In the same scenario, no experiments were able to fully identify the major attack types in real-time mode. We also found 66% of the experiment were conducted based on outdated dataset (DARPA), which may degrade IDS efficiency.
The area of big data analytics in IDS is still in its initial stages, and we anticipate major advances on the horizon. Thus, we recommend that more adaptive DMT be integrated with available big data tools. Moreover, additional effort is required to design real-time IDSs to fully classify several types of attack. Another significant point is that Spark outperforms other popular big data tools such as Hadoop, Storm, and Jubatus. In the light of such a scenario, it is recommended to consider Spark for future research with regards to IDS.