{'fraction_non_alphanumeric': 0.03654566888955361, 'fraction_numerical': 0.010363697147783858, 'mean_word_length': 5.124353448275862, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Supervised Feature Selection Techniques in Network Intrusion Detection: a Critical Review

Machine Learning (ML) techniques are becoming an invaluable support for network intrusion detection, especially in revealing anomalous flows, which often hide cyber-threats. Typically, ML algorithms are exploited to classify/recognize data traffic on the basis of statistical features such as inter-arrival times, packets length distribution, mean number of flows, etc. Dealing with the vast diversity and number of features that typically characterize data traffic is a hard problem. This results in the following issues: i) the presence of so many features leads to lengthy training processes (particularly when features are highly correlated), while prediction accuracy does not proportionally improve; ii) some of the features may introduce bias during the classification process, particularly those that have scarce relation with the data traffic to be classified. To this end, by reducing the feature space and retaining only the most significant features, Feature Selection (FS) becomes a crucial pre-processing step in network management and, specifically, for the purposes of network intrusion detection. In this review paper, we complement other surveys in multiple ways: i) evaluating more recent datasets (updated w.r.t. obsolete KDD 99) by means of a designed-from-scratch Python-based procedure; ii) providing a synopsis of most credited FS approaches in the field of intrusion detection, including Multi-Objective Evolutionary techniques; iii) assessing various experimental analyses such as feature correlation, time complexity, and performance. Our comparisons offer useful guidelines to network/security managers who are considering the incorporation of ML concepts into network intrusion detection, where trade-offs between performance and resource consumption are crucial.


Introduction
With the rapid growth of digital technology and communications, we are overwhelmed by network data traffic, which are diverse for media type (e.g. video, voice, text, sensory, etc.), and originate from (and are transported through) a broad range of sources (e.g. mobile networks, cloud infrastructures, Internet of Things, etc.). Consequently, we handle high-dimensionality data, calling for increasingly more sophisticated classification methods [1,2].
Typically, we refer to high dimensionality when we deal with data whereby a large number of features may be extracted, to the point that the features may even exceed the number of observations. This leads to major issues, particularly the massive increase in training times.
To this end, Feature Selection (FS) is a promising research direction, looking at ways to reduce the feature space in order to pinpoint only the most significant features. As a fundamental pre-processing step in machine learning, FS is gaining prominence in network management and, specifically, for the purposes of network intrusion detection and network traffic classification problems [3,4,5,6].
Machine learning engines can be easily embedded in network intrusion detection systems (IDS), which represent an essential part of network infrastructures to guarantee security [23,24,25] and availability [26,27,28,29,30,31,32,33,34]. Specifically, modern NIDS can be equipped with software probes in charge of analyzing network traffic on the basis of some characterizing features such as: distribution of inter-arrival times, distribution of packet sizes, presence of specific TCP/IP flags, percentage of forward/backward flows. Such statistical information is instrumental to revealing anomalous traffic, which is often behind Distributed Denial of Service (DDoS) attacks [35,36], covert Voice-over-IP sessions [37], threat diffusion [38] and Peer-2-Peer traffic [39,40,41]. In many cases, these flows would pass unobserved through other conventional signature-based analyses.
In principle, a larger number of features would allow to perform more granular analyses; yet, two main drawbacks emerge. First, a proliferation of correlated features leads to levels of redundancy that are useless, while resulting in increasingly longer training times. Also, not all features are valuable in characterizing traffic, incurring bias during the classification step.
If we consider, as an example, some novel probes such as ISCXFlowMeter [42], these can actually generate more than 80 features to characterize data traffic, which makes it ex-tremely difficult for the network analyst to deal with this vast amount of information. This is where an FS pre-processing step becomes invaluable.
Across the scientific literature, many works are devoted to surveying machine learning algorithms with applications to internet traffic classification. Yet, few attempts have been made in the field of FS techniques, where two main shortcomings emerge: i) many works analyze and compare FS algorithms applied to non-specific datasets [43,44,45]; ii) are either based on outdated datasets, or lack the experimental dimension. Overall, not much novel algorithms have been compared (refer to details provided in Table 1, discussed further ahead). Aimed at filling these gaps, our paper provides the main following contributions: • We perform an experimental analysis of more recent datasets (classic literature focuses on the 20-year old KDD99 dataset) by means of our designed-fromscratch Python based routine. This allows us to i) perform cleaning, re-balancing, and data mixing operations, and ii) automatically interact with specific MLbased engines.
• We present a variety of experimental results (including: feature correlation, time complexity, performance) aimed at critically comparing selected families of FS algorithms, ranging from classic ones (rank search, linear forwarding selection) to modern bio-inspired algorithms (genetic search, ant colonies, multi-objective evolutionary), thus going well beyond a conventional literature survey.
Our experimental-based, comparative assessment finds fruitful applications in the field of network/security management, where machine learning techniques are proved to offer a precious support to intrusion detection tasks, and where critical trade-offs between performance classification and resource consumption arise. The paper is organized as follows: Section 2 offers a general perspective on FS methods, in line with other accredited taxonomies. In Section 3 we analyze how diverse FS techniques have been applied in the literature, often based on the old KDD99 dataset. Section 4 proposes an excursus of popular FS algorithms (classic and modern) along with some necessary technical details. In Section 5 we analyze the exploited novel datasets by grouping the most relevant features in families. In Section 6 we perform an experimental-based comparative analysis of prominent FS algorithms, providing performance, feature correlation, and time-complexity figures. Finally, Section 7 draws conclusions and provides future-direction indications.

Overview of Feature Selection
Feature Selection refers to that set of techniques and strategies that allow to optimize the feature space, namely, an -dimensional space where each sample is represented as a point. When dealing with large feature spaces, the analysis of data, which starts from their representative features, can be tremendously time and resource consuming. Hence, the need to devise suitable FS strategies aimed at eliminating ) irrelevant features, namely, those features that are not actually needed to build an optimal feature subset; and ) redundant features, namely, those features that strongly depend on other features [46].
Feature selection can be considered as a special case of feature extraction methods [47]. The latter refer to a set of techniques (e.g. PCA, Single Value Decomposition, Linear Discriminant Analysis and others) useful to transform the original feature space in a new one aimed at alleviating the effects of the notorious curse of dimensionality problem [48,49]. It is important to note that feature extraction comes with the critical risk of obtaining a transformed feature space that could lose its original physical meaning, whereas classic FS aims at preserving it [50,51].
A more formal definition of the FS problem follows. Given a feature set = { ∶ = 1, … , }, find a subset where an objective function Y(⋅) is optimized, namely: Unless otherwise stated, the problem is to select the optimal subset of features (according to a specific criterion) from the initial set, where two steps are typically performed: the first one involves a search strategy to pinpoint candidate subsets; the second one involves an objective function to evaluate the selected candidate subsets. The latter can be split in two types [52,53,54,55]: ) filters, referring to objective functions that rely on properties of the data by evaluating the information content (e.g. correlation measures, inter-class distance); ) wrappers, referring to objective functions that exploit training models by starting from a subset of features and, then, adding or removing features based on the previous model. A commonly accepted taxonomy of FS methods is presented in [56], where three classic approaches have been identified as supervised, unsupervised, and semi-supervised. According to the supervised approach, labeled data are exploited to single out a feature subset, considering specific criteria for measuring the features importance. Conversely, unsupervised techniques seek to unveil the intrinsic data structure to select the most significant features, without assuming any a priori knowledge [57].
Finally, the semi-supervised approach is based on a mixed strategy, striving to enrich an unlabeled set with some labeled data, so as to improve the FS phase. Both the unsupervised and the semi-supervised approaches exhibit the drawback of neglecting potential correlations among features, resulting in the analysis of sub-optimal sets. This may prove critical when dealing with traffic analysis, where we need to take into account statistical-based features (e.g. inter-arrival times variance, average packet length, etc.) and deterministic ones (e.g. IP addresses, port numbers, etc.). Let us consider, for example, a particular kind of traffic directed towards a fixed destination port for a certain period of time. An unsupervised approach could lead to a subset of features which does not include the destination port. Thus, a crucial (as well as deterministic) piece of information gets lost.
On the other hand, a supervised approach can offer optimal results, provided that the data are correctly labeled. This case typically occurs in a controlled network environment, where, with the help of network analyzers, it is possible to automatically label the type of passing data traffic.
Because of the great potential of supervised methods, and thanks to the availability of suitable labeled datasets, we have decided to focus our experimental-based comparative evaluation on supervised FS methods.

Related Work on Feature Selection applied to ML-based Intrusion Detection
Most scientific literature involving ML approaches is typically focused on the proposal of algorithms or techniques for classification/detection of specific network traffic ( [67,68,69,70,71,72,73,74]). Unfortunately, the straight application of machine learning to network traffic analysis is not always feasible, due to the broad variety of traffic types, which leads to unmanageable feature spaces. Intuitively, in fact, a highly diversified traffic (multimedia, asynchronous, bursty, etc.) requires a large set of features able to capture the variegated "nature" of the different flows. That is why appropriate pre-processing steps (i.e. FS) play a crucial role, thus a significant portion of ML-based literature has shown interest in FS techniques.
For instance, to improve the performance of IDS frameworks, the authors of [75] propose a mixed strategy involving Principal Component Analysis and fuzzy clustering with KNN-based FS techniques. A correlation-based FS approach coupled with a Support Vector Machine (SVM) classifier is proposed in [76] to build a cloud-based IDS. An IDS based on deep learning methods, along with a filter-based FS algorithm, is introduced in [77]. Similarly, a Convolutional Neural Network (CNN) approach is exploited in [78] to select traffic features from raw data sets, improving the accuracy of an intrusion detector. Yu and Liu propose a mutual information-based algorithm that can analytically select optimal features for classification, by handling linearly and non-linearly dependent data [79]. Again, FS is exploited jointly with Artificial Neural Networks [80] and Deep Neural Networks [81], respectively, to improve IDS performance. Furthermore, authors in [82] embed in an IDS two FS algorithms that are compared against mutual information-based methods.
A major shortcoming of these works is that methods are validated and compared through the outdated KDD99 dataset. This contains information about older network attacks (that have been mitigated by now) or, in some cases, adopts an updated version of KDD99, namely NSL-KDD [83]. Yet, although NSL-KDD adds some improvements onto KDD99 (e.g. no redundant records, better balancing between training and test set, etc.), it does not take into account features that characterize novel cyber attacks. Other works in the field of machine learning applied to intrusion detection rely on more recent datasets such as UNSW-NB15 [84,85,86,87]. Although quite recent, such a dataset has two limitations: first, the traffic has been collected in a reduced testbed; and secondly, the number of features is limited to 49, which is too small to appreciate the effectiveness of feature selection techniques. In contrast, in our work we rely on an up-to-date dataset from the Canadian Institute for Cybersecurity [88] which has the following benefits: ) it contains data traffic gathered over a vast network area; and ) it accounts for about 80 features, allowing to extensively test the feature selection algorithms. Additional details about this dataset are provided in Section 5. Going more specifically into the set of works that share with this paper the aim to compare or survey FS methods for intrusion detection, we have collected the significant papers in Table 1. In it, we have identified the material covered in the literature, which helps appreciating the contributions of our paper. The first column of Table 1 points to the source; the second column highlights the type of experimental analysis (if any); the third column pinpoints the type of datasets utilized (single or multi-class); the last column provides a concise description of the surveyed material.

Review of Feature Selection Algorithms under Scrutiny
In this section, we briefly describe the algorithms under scrutiny, which belong to different families of FS techniques, ranging from classic rank-guided (Rank, Linear Forward Selection) and meta-heuristic (Tabu, Scatter, Particle Swarm) ones, to nature-inspired algorithms (Ant, Cuckoo), and up to modern techniques (Genetic, Multi-Objective Evolutionary).
For each algorithm, we provide a brief recap along with its pertinent application in network traffic analysis and security in literature.

Rank-based Feature Selection
Algorithms belonging to this family follow an approach based on two macro-steps: in the first one, the features are ranked according to a certain statistical measure, whereas in the second step the algorithm chooses the top ranked features (eventually partitioned in clusters). We investigate and put to test two representative algorithms of this family: Rank Search and Linear Forward Selection.

Rank Search
The Rank Search technique refers not only to a single algorithm, but, to an umbrella of methods able to produce a list of attributes ranked with some criteria. One of the most applied rank-based techniques in FS relies on the Information Gain (IG) concept [89]. Given an attribute A and a class C, the entropy of the class without and with prior observation of the attribute are, respectively: and The IG is given by ( ) − ( | ), and represents the amount of the class entropy decreasing due to the a priori knowledge introduced by -th attribute.
Over network traffic analysis literature, the rank search method has been widely exploited in conjunction with standard machine learning algorithms typically used during the classification step. In [90], having the NSL-KDD dataset as input, the proposed algorithm ) evaluates the information gain value, ) applies fuzzy rules to remove unwanted features, ) calculates the mean value of IG across the new subset of features, ) refines the feature subsets by applying an algorithm based on conditional probability evaluation. Authors in [91] propose a detection model able to cope with network attacks based on the feature IG ranking; once obtained a satisfying feature set, a triangle area based KNN, combining both SVM and greedy techniques, is exploited to single out even more discriminative and useful features. A combination between an IG-based feature selection method and a C4.5-based classifier is advanced in [92], where the resulting algorithm has been optimized (in terms of power consumption) to reveal DoS attacks in ad hoc networks. Similar ensemble has been exploited in [93], where authors consider a broader set of classifiers (C4.5, Random Forest, Naive Bayes, etc.) and where the analysis focuses on generic malware detection. Again, a mixed Genetic/KNN approach for intrusion detection presented in [94] benefits from a feature reduction procedure obtained applying ID3 algorithm to find higher IG.

Linear Forward Selection
Such a technique to reduce the feature space dimensionality has been presented in [95]. Linear Forward Selection (LFS) can be considered as an improvement of Sequential Forward Selection (SFS) method which starts with an empty set of features and sequentially adds one feature at a time. In order to face the ( 2 ) complexity of SFS, in LFS the number of considered features at each step does not exceed a certain user-specified constant, thus, the resulting performance is impressively ameliorated. Two methods for limiting the number of features have been implemented in LFS algorithm: ) Fixed Set, where a score obtained by means of a wrapper evaluator is used to choose the top-ranked attributes, thus, the maximum number of evaluations reduces to ∕2( + 1) ; ) Fixed Width, where, at each forward selection step, the number of features is increased by one, such that the set of candidate expansions consists of the best features not been selected so far during the search. In such case, the maximum number of evaluations amounts to ⋅ − ∕2( +1). Forward Selection-based techniques have been exploited across the traffic classification domain for detection of malicious application on Android [96], detection of zero day attacks [97], designing novel anomaly detection systems [98].

Meta-heuristic Feature Selection
Techniques belonging to this family rely on the principle that it is possible to select heuristics, namely, approximate algorithms searching for a sufficiently good solution to an optimization problem, useful when some constraints arise (e.g. limited computational resources, incomplete information). We select three representative algorithms of this family: Tabu Search, Scatter Search, and Particle Swarm Optimization.

Tabu Search
Tabu Search (TS) algorithm has been originally proposed in [99], and, then, it has been extended and exploited to solve practical optimization problems [100,101,102]. TS is based on a metaheuristic method able to pilot a local heuristic search in exploring the space of solutions beyond the local optimality. Two main features characterize TS: adaptive memory and responsive exploration. The former allows to perform local choices guided by information collected during the search, whereas, the latter allows to make strategic choices. In other words, TS exploits a local search procedure combined with memory-based strategies, thus, the issue of getting trapped in local optimal solutions is avoided. To implement the memory-based mechanism, TS builds a map of recently visited solutions called Tabu List (TL). A simplified TS algorithm is illustrated below: Algorithm 1: Tabu Search 1. Given a function ( ) to be optimized over a set , start from initial solution 0 ∈ , initialize Tabu List (TL), and initialize a counter = 0.

Given
can be reached by by means of a move operation. Thus, generate a neighborhood move list ( ). 3. Given +1 the best solution in ( ), update TL. 4. In case stopping conditions are met, terminate.
It is interesting to notice that, the memory structures characterizing TS method operate according four dimensions [100]: • Recency: concerns the ability of keeping track of solutions attributes that have changed across the recent past.
• Frequency: involves the mechanism exploited to broad the foundation for selecting preferred moves.
• Quality: pertains to the capacity of discriminating the merit of solutions visited during the search operation.
• Influence: takes into account the impact of choices performed during the search.
In the field of network traffic classification, TS has been exploited in [103] jointly with fuzzy techniques aimed at optimizing the exploration of feature search space in intrusion detection problems. A combination of TS technique and KNN is presented in [104], where KNN is initially exploited to generate a subset of non-redundant features, whereas, TS is used to refine the obtained subset. Again, authors in [105] propose a wrapper FS algorithm, dubbed GATS-C4.5, that embeds an hybrid Genetic and Tabu-based method as feature selection strategy and a supervised ML algorithm (C4.5) as the evaluation function. Similar combinations between TS and Genetic techniques have been used in [106] where some tests (vs pure Genetic algorithms) across DARPA dataset have been carried out, and in [107] where SVM has been adopted as a classification criterion.

Scatter Search
Scatter Search is a metaheuristic algorithm involving memory-based mechanisms similar to those exploited in Tabu Search [108]. The main strategy relies on an iterative process that organizes high-quality optimal solutions into subsets, and where five "methods" emerge: ) Diversification, to create a set of different trial solutions by exploiting a seed solution; ) Improvement, to convert a trial solution in one or more improved solutions; ) Reference Set Update, to create and keep update a reference set of best solutions; ) Subset Generation, to manipulate the reference set aimed at deriving a subset of solutions; ) Solution Combination, to linearly re-combine the solutions on the basis of subsets obtained with the previous methods.
As an effective FS procedure, Scatter Search has been exploited together with NLP solvers for global optimization [109], with rough sets for the implementation of credit scoring mechanisms [110], or in a parallelized fashion to improve the feature subset selection problem [111]. Again, Scatter Search has been profitably exploited in issues involving credit cards fraud detection [112], and software security characterization [113].

PSO Search
The Particle Swarm Optimization (PSO) algorithm has been originally discovered in [114], through simulations carried out across a simplified social model aimed at reproducing the behavior of birds flocking.
Then, the population of agents became more similar to a swarm than flock, and single individuals were named particles that represent the candidate solutions. In a mathematical form, given  ⊂ the search space, and ∶  ← ← → ⊆ , the swarm is defined by a set It is assumed that particles are able to iteratively move within search space  by means of a velocity parameter defined as = ( 1 , 2 , … , ) with = 1, 2, … , . In PSO method, it is also defined a memory set = { 1 , 2 , … , } containing the best positions = ( 1 , 2 , … , ) (with = 1, 2, … , ) visited by each particle. Given the time counter, the current position and velocity for particle are, respectively, ( ) and ( ), whereas ( ) = arg min ( ). Accordingly, PSO is defined by the following equations: where: 1 and 2 denote random variables uniformly distributed in [0,1], 1 is the cognitive parameter which affects the step size that the particle takes towards its best candidate solution ( ), and 2 is the social parameter which affects the step size that the particle takes towards the swarm's best solution ( ). At each iteration, best positions ( + 1) are updated as well, namely In the field of FS applied to network traffic analysis, PSO has been profitably exploited jointly with classification techniques. It is the case of [115,116], where a particle swarm selection method has been exploited with SVM-based classifiers. Again, a hybrid FS model based on PSO and Random Forest has been exploited in [117], where independent measures and a learning algorithm are exploited to evaluate feature subsets. Interesting is also an evolution of classic PSO advanced in [118], and dubbed Accelerated PSO, amenable to deal with FS on Big Data streams.

Nature-inspired Feature Selection
Although relying on meta-heuristic concepts, this family of algorithms takes inspiration from the nature ecosystem, where many animal species exhibit impressive behaviors aimed at optimizing their life cycle. We assess Ant Optimization and Cuckoo search as representative algorithms of such family.

Ant Search
This technique, originally proposed in [119], is inspired by ants colonies behavior, where the optimization problem is solved by a "colony" of cooperating agents. Analyses carried out by ethologists showed that, following pheromone trails, each ant is able to follow a preceding ant which releases such substance, thus, a whole ant colony is able to self-organize itself. The emerging collective behavior relies on a positive feedback loop: the probability which an ant chooses a certain path increases with the number of ants that choosing the same path, since the trail is continuously reinforced with new pheromone. The Ant algorithm can be profitably exploited in feature selection problems by evaluating the probability that the -th "ant" could arrive to feature by starting from feature , namely, where, is the set of feasible attributes (not visited yet), is the amount of pheromone across the path, represents the heuristic information for the selected attribute , and are parameters in charge of controlling pheromone trials and heuristic information, respectively.
As regards the FS problem in network traffic analysis, Ant-based methods have been used in: [120] where a SVM classifier is adopted on KDD99 dataset, after a feature reduction obtained through ACO (Ant Colony Optimization) technique; in [121] where an ACO-based feature selection method allows to deal with big streamed data; [122] where an improved ACO-based algorithm (named FACO) has been designed and tested across the classic KDD99 dataset.

Cuckoo Search
This technique is inspired to the brood parasitism strategy characterizing some cuckoos species. In particular, such species lay their eggs in the nests of other birds (hosts). Since host birds can engage a conflict as they recognize alien eggs, particular cuckoo species have evolved in such a way that females are able in mimicking colour and size of eggs of some host species, thus the hosts are cheated, and the probability of cuckoos reproductivity grows. Considering an egg in a nest as a solution, three idealized rules emerge in Cuckoo Search procedure [123]: ) each cuckoo lays one egg at time, in a randomly chosen nest; ) the bests nests (having highquality eggs) are candidate to carry over next generations; ) the number of nests is fixed and, as a host bird discovers alien (cuckoo) eggs with a probability , it gets rid of it. The aim of the algorithm is to exploit new (and eventually better) solutions in place of not-so-good solutions in the nest. New solutions +1 for cuckoo are obtained through the following expression which represents the stochastic equation for random walk, namely, where, > 0 is a scale factor, the product ⊗ refers to entrywise multiplications, whereas ℒ is the Lévy distribution with (1 < ≤ 3).
Cuckoo search method has been exploited in network traffic analysis paired with various techniques and technologies. In [124], authors propose an algorithm that uses PCA and Cuckoo Search to reduce the feature space and to optimize the clustering center selection. A Cuckoo-based FS algorithm is proposed in [125] to preprocess network data aimed at improving the IDS detection accuracy in cloud environments. A Cuckoo search strategy has been also used in [126] to optimize Artificial Neural Networks when dealing with traffic anomaly detection issues. Again, coupled with SVM, Cuckoo search has been adopted in FS to deal with problem of phishing mail detection [127]. Recently, extended versions of Cuckoo Search algorithm have been advanced to cope with classification of tweets in sentiment analysis [128], or to defeat attacks in Software Defined Network infrastructures [129].

Evolutionary Feature Selection
Such family of algorithms is inspired by natural selection theory, claiming that living organisms survived across millions of years thanks to an adaptation process. In a similar way, this aptitude can be translated in search for optimal solutions to a problem. Two exemplary tested algorithms are: Genetic search and Multi Objective Evolutionary search.

Genetic Search
Genetic Algorithms (GAs) have been designed around the mid-1950s, when biologists started to perform computerbased simulations aimed at analyzing more in deep the evolution of genetic processes [130]. Then, GAs have been extended to face problems ranging from neural networks weight estimation [131] to inequalities-based problems [132]. A pioneering work in this field has been carried out by Holland [133,134], and, today, many variants of GAs exist [135] and are applied in economy, computer science, sociology.
The basic skeleton of a GA includes three operators [136]: Reproduction, Crossover and Mutation.
Reproduction refers to a process in charge of evaluating the ability of an individual to be selected (among others) for reproduction, on the basis of a fitness score.
Crossover concerns the capability of a genetic operator in recombining information to create new offspring. Typically, offspring is generated by exchanging genes of parents until a crossover point is reached.
Mutation pertains to the probability that some offspring genes could be modified or altered.
Genetic-based feature selection in network traffic analysis has been used in conjunction with many ML-based methods. Authors in [137] exploit a GA-based FS approach to optimize network traffic data before applying an artificial neural network to perform attacks detection across cloud infrastructures. A combination of a genetic FS method and a supervised classifier based on J48 algorithm is proposed in [138]. More frequent across the scientific literature is the coupling between genetic FS and SVM classifiers applied to network traffic classification problems (see [139,140,141]).
When dealing with FS problems, GAs allow to explore the solution space by selecting the most promising regions, thus, avoiding a costly exhaustive search. In our domain, the initial population is represented by the whole feature space and the fitness function relies on the correlation among fea-tures and expressed by means of a merit indicator defined further ahead in eq. (12).
Once entered the cycle represented in Fig. 1, the algorithm calculates the fitness of each candidate solution per iteration, selects individuals to reproduce, and generates a new population by taking into account crossover (feature recombination with a certain probability), and mutation (one feature can be turned into another feature with a certain probability).

Multi-Objective Evolutionary Search
The family of solutions concerning a multiobjective optimization problem (MO) includes all the elements of the search space whose objective vectors cannot be simultaneously improved (Pareto optimality concept) [142]. The set of such objective vectors is said non-dominated.
More formally, a MO problem can be formulated as follows: given a vector of objective functions of a vector variable in a domain  defined as a decision vector ℎ ∈  is Pareto-optimal iff there is no ∈  such that: On the other hand, Evolutionary Algortihms (EAs) can be profitably exploited in MO-based problems since many "individuals" can search in parallel for multiple solutions, with the possibility of taking advantage of similarities among solutions belonging to the same family. Possible implementations of MO-EA techniques are ENORA (Evolutionary NOn-dominated Radial slots based Algorithm), and NSGA (Non-dominated Sorted Genetic Algorithm) compared in [143].
Applied to the FS problem, the purpose of a multiobjective search algorithm is to discover a subset of features (a family of solutions) being a good approximation of the Pareto front. The MO-EA approach has been exploited in network traffic classification jointly with ensemble MLbased methods [144], where some objectives such as maximizing true positive rate, maximizing classification accuracy, minimizing feature number, and minimizing false positive rate, are satisfied simultaneously and with no conflicts. An NSGA-based approach to ameliorate the performance classification of IDS platforms has been adopted in [145] and in [146]. Again, a MO-EA technique jointly with fuzzy classifiers for coping with the traffic classification problem has been introduced in [147].

The considered Datasets
One of the great issues when dealing with supervised FS approaches in traffic analysis, is finding training sets that are both recent and labeled. As remarked in Sect. 3, many works rely on the obsolete KDD99 dataset [148]. Created about 20 years ago, KDD99 has been broadly employed to validate machine learning algorithms, particularly to differentiate malicious data traffic from benign one. However, the KDD99 dataset does no longer reflect the characteristics of modern data traffic, which has sensibly changed across time. This is very much the case of multimedia traffic (e.g. voice, video), being the principle revenue-making stream for service providers but also the vehicle of covert malicious data. Per contra, in this work we consider more recent datasets obtained by means of CICFlowMeter [42,149], an open-source engine that can both gather and label network traffic in a controlled environment. Each dataset contains records labeled either as Benign or Malicious (malicious traffic is often split in sub-labeled traffic representing different kinds of attacks). Specifically, we consider the following datasets: • DDoS which contains traffic relating to distributed denial of service attacks, aimed at saturating the network resources of specific targets; • Portscan which contains traffic relating to Portscan attacks, aimed at discovering open, network device ports; • Webattack, including malicious traffic which implements various web-based attacks such as Brute Force, Cross-Site Scripting, and Sql Injection; • TOR which includes traffic passing over the TOR network, an anonymous and private data circuit often exploited to carry dangerous information or malicious encrypted traffic; • Android which embeds various families of Androidbased threats (adwares, ransomwares, etc.).
We want to notice that the first four datasets are exploited into the single class analysis, whereas the Android dataset is exploited in the forthcoming multi-class analysis. In this latter analysis, we build a new dataset called MultiAndroid (see Sect. 6.2), obtained by selecting the most relevant mobile threats from the Android dataset mixed with some benign traffic. In order to avoid the issue of bias that typically arises during classification in imbalanced datasets, we have first re-arranged the datasets. We have achieved wellbalanced benign/malicious features, spanning across about 50 instances for each dataset. Each one contains up to 78 features, except for the TOR dataset including 30 features. For the sake of convenience, we found it useful to group the features in 5 macro-classes (the complete list of features can be found in [150]): •

Experimental Results
Herein we describe our analytical study, which required the development of a dedicated Python routine, to normalize/balance the datasets and to automate the comparison among FS algorithms. We have validated the scrutinized search algorithms using the Correlation-based Feature Selector (CFS) as objective function [151,152].
The main idea behind CFS is that a good feature subset includes those features that are highly correlated with the class, while being strongly uncorrelated among them. A formal definition is offered in [153]: a feature is relevant iff there is some and for which ( = ) > 0 such that Namely, is relevant if is conditionally dependent on . Thus, CFS is a filter algorithm that can rank feature subsets according to a correlation-based heuristic function. Precisely, given a subset including features, the heuristic merit , is defined as: where is the average value of feature/class correlations, and is the average value of feature/feature correlations. The numerator of (12) may be seen as an indicator of how far a set of features is predictive of a class; whereas, the denominator contains information about how much redundancy there is among features.  Our assessment is split into two parts: the first one concerns a single class analysis, where we evaluate datasets exhibiting dichotomous information (malign/benign); the second one is focused on multi class problems, where we evaluate the effectiveness of FS in the presence of multiple classes.

Single Class Analysis
Let us consider the Distributed Denial of Service (DDoS) attack which, recently, is also affecting modern SDN-based networks [154,155]. DDoS attacks are designed to overwhelm the target network resources by means of a botnet, namely, a network composed of a large number of malicious nodes sending tiny packets towards the target, ultimately coordinated by a botmaster.
Let us now analyze the results obtained by preprocessing the DDoS dataset through the set of FS algorithms introduced above. In Fig. 2 we report, for each algorithm, the correlation map corresponding to a graphical representation of covariance matrices. This representation embeds three important pieces of information: i) the number of features surviving after the FS processing step; ii) the type Training Size   of features; and iii) the relationship existing among surviving features. The latter is taken into account by means of a gray scale, in which darker shades indicate higher levels of correlation. Thus, each ( , ) "pixel" gives the correlation level between feature and feature . Accordingly, the pixels on the main diagonal are always black (maximum correlation, corr=1), due to the self-correlation. As was to be expected, higher correlation are found among those features belonging to the same family (Time-based, Flow-based, etc.). Some interesting considerations about the various correlation maps arise. First, the number of features retained by different algorithms may significantly diverge, which is due to the specific approaches adopted by each algorithm. The Genetic algorithm is the one retaining the most features. This is to be ascribed to the particular strategy of this algorithm, which strives to escape local optima by applying the mutation operator, thus allowing to consider more paths, namely, more features. Second, some common features retained by all the algorithms can be recognized. For instance, the destination port feature is always present since, in a DDoS attack, a target victim is typically reached on a particular exposed TCP/UDP port. Moreover, since DDoS attacks are characterized by a large amount of small-size packets, features embodying information about packet lengths are retained. The difference is that, some algorithms (e.g. Scatter, MO-EA, Cuckoo, Tabu, LFS) just keep the essential features related to packet length (e.g. total packet length, total number of bytes sent in initial window); whereas, other algorithms (e.g. Ranking, Genetic, PSO, Ant) prefer to retain more features belonging to the same family. DDoS is also characterized by some kind of synchronization among the bots, which are coordinated to launch an almostsimultaneous attack. This means that time-related features will often provide useful information to detect DDoS. Interestingly, the Genetic algorithm retains 5 features relating to the inter-arrival flow times, resulting in a dark gray cluster at the center of the correlation map ( Fig. 2(g)).
It is also possible for DDoS attacks to be even more effective through the modification of the IP flags (e.g. SYN/RST flooding). Accordingly, features embodying information about IP flags (e.g. RST-SYN-URG flag count) are retained by algorithms such as Ant (Fig. 2(a)), MO-EA (Fig. 2(c)), Cuckoo (Fig. 2(e)), Genetic (Fig. 2(g)), and PSO ( Fig.  2(h)). Let us note that many algorithms opt for selecting features that are uncorrelated among them (few dark gray or black clusters are present) since they convey more variegated information.
Let us now analyze some findings obtained from the time-complexity evaluation. To this aim, we use a PC equipped with Intel Core TM i5-7200U CPU@ 2.50GHz CPU and 16 GB of RAM. In Fig. 3(a), we show how the FS time varies with training size, for the DDoS dataset. No dramatic differences are observed across the various algorithms, even more significantly as the training size grows. Considering a relatively large training size (with 5 ⋅ 10 4 training instances), FS times range from about 10 seconds (Scatter algorithm) to almost 26 seconds (MO-EA algorithm). Surprisingly, the FS times are rather uniform, in spite of the broad variation in number of retained features (by each of the algorithms). For instance, remaining in the case of 5 ⋅ 10 4 training instances, Scatter retains the minimum number of features (4), while Genetic retains the maximum number of features (27); yet FS times are comparable (16.19 and 10.18 seconds, respectively). Although it is legitimate to expect that higher FS time could be justified to produce a more reduced feature space, the scarce correlation between such observables is due to the particular logic implemented in each FS algorithm.
On the other hand, Fig. 3(b) provides the training times obtained by applying the J48 benchmark algorithm, downstream of the FS processing step. Here, the black line (with empty circles) gives the training times obtained when no FS processing is employed. We can observe how FS leads to significant improvements, in terms of both times and trends. The black (benchmark) line grows rapidly to almost 80 seconds, while most algorithms peak to almost 5 seconds, with the exception of the Genetic algorithm (yellow line) and the Particle Swarm algorithm (light blue line) that take over 10 seconds to complete. This indicates that the FS process, on the whole, brings gains in the range of about one order of magnitude, which may become even more significant as the dataset grows.
Let us now analyze the performance of the proposed FS algorithms in terms of Accuracy and F-Measure. These two metrics, widely used in the field of traffic classification [156,157], are defined as follows: • Accuracy: the ratio of the correctly predicted observations to the total observations. This is the most intuitive indicator.
• F-Measure: the weighted average of precision (ratio of correctly classified flows over all predicted flows in a class) and recall (ratio of correctly classified flows over all ground truth flows in a class). This is an indicator of a per-class performance.
To verify that the effectiveness of the FS algorithms is not linked to specific datasets, we have considered the 4 different datasets introduced in Sect. 5 (DDoS, Portscan, We-bAttack, and TOR), reporting our findings in Fig. 4. Just like for the previous experiments, we have used the treebased J48 algorithm as a benchmark. We have adopted a 10-fold cross-validation which is typical in applied ML, and offers a good trade-off between training time and robustness. Noticeably, all FS algorithms perform satisfactorily (both in accuracy and F-measure) in comparison to the benchmark (first bars in all the histograms, labeled as "NO F.S.") for the four datasets.
In some instances the FS algorithms performed even better than the benchmark (e.g., Rank and Genetic algorithms in the WebAttack dataset). This can be explained by a phe-   nomenon that is well-known in ML, whereby models based on too many features may lead to biased classification. On the other hand, when FS manages to retain a sufficiently high number of meaningful features, there is a positive effect on accuracy. This is the case of the Genetic algorithm applied to the TOR dataset ( Fig. 4(d)) that performs better than the other methods.

Multi Class Analysis
Another fruitful analysis is aimed at evaluating FS algorithms when multi-instance datasets are considered. This turns out to be particularly useful when it is not possible to discern different types of data traffic via some pre-processing filter (e.g. IP/Port-based filtering). To assess this case, we consider two datasets: the MultiAndroid dataset, containing benign traffic mixed up with five different types of Androidbased threats; and the DDoS/Portscan dataset, including a Training Size Training Size mix of DDoS, Portscan, and benign traffic. The MultiAndroid dataset, includes the following types of malign traffic: • FakeApp.AL: a scareware hidden inside a fake Minecraft application, one of the most popular game applications; • Android Defender: a malware which, once accidentally downloaded and installed, raises some fake alerts; • Gooligam: an insidious malware that has already infected more than 1 million Android-based devices, aimed at stealing Google accounts for Drive, Docs, Gmail, etc.; • Feiwo: belonging to the adware family, it acts by showing advertisements in the system notification bar, and by sending device GPS coordinates to a remote server; • Charger: a ransomware hidden in some Google Play applications, which gains root privileges and steals contacts before asking for a ransom.
Let us analyze how FS algorithms impact on the Mul-tiAndroid dataset in terms of feature correlation referring first to the panels of Fig. 5. Comparing these results with the ones of Fig. 2, an interesting difference emerges: all FS algorithms retain more features w.r.t. the single-class case. This behavior is coherent with the fact that, to deal with different types of threats (ransomware, adware, malware) we need more features, to be able to capture this higher variability. This effect is even more evident in time-based features (mainly inter-arrival times) and in size-based features (mainly packet lengths).
Looking at DDoS, we observe a difference between single-and multi-class analysis. In the latter, the destina-tion port is not retained as a crucial feature. This is possibly because malwares exploit different mechanisms to create damage: rather than directly overwhelming a particular target port, they first act in the background (e.g. by stealing privacy data) and then produce malicious traffic in egress. On the other hand, DDoS attacks generate ingress traffic from the infected device.
It is worth noticing that, when applied to multi-class problems, all algorithms have preserved their original logic. For instance, with 31 surviving features, the Genetic algorithm is still the algorithm that saves more features, thanks to the role played by the mutation operator. Another example is the MO-EA algorithm that, just like in the single-class experiment, retains the smallest number of features (6). This is mainly due to the diversity-preservation mechanism, which forces the selection of a representative subset of the whole Pareto front. It optimizes conflicting objective functions, thus few solutions survive.
The time-complexity evaluation is reported in Fig. 6, which evaluates the usual FS algorithms onto the MultiAndroid dataset. FS times exhibit the same order of magnitude as in single-class analysis (Figs.3(a)). For a training size amounting to 5 ⋅ 10 4 instances, the fastest algorithm is Scatter (FS time amounting to 9.541 seconds); whereas the slowest one is MO-EA (FS time amounting to 24.827 seconds).
The situation changes dramatically when we consider training times for the J48 benchmark algorithm (Fig. 6(b)). Notably, multi-class algorithms are roughly one order of magnitude slower than their single-class counterpart. For instance, let us consider the Genetic algorithm (yellow curve). cant gains are still obtained by all FS algorithms compared to the "NO F.S." benchmark, which peaks at 446.329 secs.
Turning now to the performance analysis, in Fig. 7 we compare the two multi-class datasets, MultiAndroid and DDoS/Portscan, drawing some interesting considerations. It is comparably more difficult to detect Android threats than DDoS/Portscan attacks -MultiAndroid accuracy is below 0.7 and F-Measure is below 0.5. However, this issue is not generated by the FS processes, since the "NO F.S." performance is poor too, particularly with the "Benign" class. This issue arises from two facts. First, mobile network attacks are often accompanied by activities that do not directly/immediately generate network anomalies. Examples are ransomware and malware, whereby the anomalies arise after the user has downloaded the malicious application. There is typically a lag between infection and anomalies, as the malicious program initially establishes a secret/silent communication with a remote server, and then gradually steals/sends private user data. Another example is adware, where those annoying banners actually incur very little data, thus making it hard to detect from the regular traffic. A second reason for the poor MultiAndroid performance is the strong similarity among different malign classes (e.g., scareware, adware, ransomware). Similar considerations hold true in the case in which we consider a dataset including Webattack and TOR traffic (not reported for space constraints), whereby the high similarity between the two classes resulted in poor classification performance. We should however stress that FS algorithms are still very beneficial, since the time-complexity benefits identified are achieved with no dramatic loss in accuracy.
By contrast, the DDoS/Portscan multi-class case achieves outstanding performance (Figs.7(c) and (d)). This is because these types of attacks are radically distinct in the way they exploit network vulnerabilities: DDoS falls under the umbrella of volumetric attacks; whereas Portscan attacks employ monitoring strategies to unveil possible open ports. In other words, a peculiar symptom of a DDoS attack is the presence of an exceptionally large number of connections coming from different nodes and heading towards one network target's port. Conversely, a symptom of Portscan attacks is the presence of just a single node (or a few nodes in case of simultaneous Portscans) opening a considerably large number of connections towards multiple ports of a certain network target. Thus it is relatively easier to differentiate between these two attacks.

General Remarks
Overall, we can observe that FS algorithms do lead to an effective reduction in feature space, ranging from 65% (Single Class, Genetic) to 95% (Single Class, Scatter) and from 60% (Multi Class, Genetic) to 92% (Multi Class, MO-EA). Such feature-space reduction translates into significant computational-time improvements, which become even more remarked as the training size grows. For instance, with a training set of 50 samples (single-class DDoS) the MO-EA algorithm takes 24.8 secs to perform FS, while the training time compared to the benchmark drops from 72.2 to 5.13 secs. At the same time, performance is not significantly degraded by the feature reduction process -accuracy drops from 0.9993 to 0.9971. Similar considerations hold for all other algorithms.
The performed assessment provides invaluable guidelines for network/security management practitioners dealing with traffic classification problems. Our evaluation framework aims at weighing the practical benefits of the various FS techniques in terms of time-complexity reduction and performance guarantees. For instance, if we aimed at minimizing the overall processing time (i.e., FS plus training times), the Scatter algorithm would be the best choice. This incurs a total processing time amounting to 14.338 seconds for the single-class case (FS= 10.178 secs plus train-ing= 4.16 secs), and to 219.963 seconds for multi-class (FS= 9.541 secs plus training= 210.422 secs). Conversely, the Genetic method would be preferable to maximize performance.

Conclusion and Future Direction
A prominent research direction for network intrusion detection is the adoption of machine learning methods, particularly for the detection of anomalous (and often malicious) network-traffic flows. Looking at the literature, we find ample examples of network classification problems. Yet, little attention has been turned towards feature selection, which is an essential classification pre-processing step. We argue that the main reason for this overlook is that most studies have been based on the obsolete KDD99 dataset, which includes few features, thus making FS irrelevant. On the other hand, we consider that modern network engines generate much richer features (in fact, hundreds of features), which allow more fine and granular network traffic analyses. However, this extra capability results into impractical ML training times, making it necessary to understand how FS may be realized effectively.
To this end, herein we have carried out an experimental comparative evaluation of prominent methods, with the view to provide insights as to how the different FS algorithms perform in the peculiar context of network-traffic classification. Our assessment shows how few, relevant features are retained, but also that the FS reduction process is virtually lossless, with a significant acceleration of the overall training process.
To sum up, the novelties of our work are: i) we carry out an experimental-based review, considering recent datasets (including DDoS, Portscan, WebAttacks, and Android threats), as opposed to the obsolete KDD99 dataset adopted in most literature; ii) we compare and contrast a representative number of alternative FS algorithm types, including classic rankguided methods (LFS, Ranking), meta-heuristic methods (Particle Swarm, Tabu, Scatter), nature-inspired methods (Ant, Cuckoo), and evolutionary methods (Genetic, MO-EA); iii) we provide actual experimental results, unveiling trade-offs between performance (Accuracy/F-Measure) and computational time, at different scales (training set size).
Ultimately, our analysis shows the benefits linked to embedding the FS process into network analysis, providing a valuable tool for identifying the most useful features out of hundreds of possibilities. This will prove invaluable to the fields of network management, security management, intrusion detection and incident response. We should note that, the purpose of our comparative evaluation was not to claim the predominance of some FS algorithms over others but, rather, to suggest a methodical framework to work with FS.
As a byproduct of our investigation, some interesting open research directions emerge: i) extending the present analysis to unsupervised FS techniques, which would be useful to deal with datasets lacking class labels, or with new types of (unknown) malicious traffic -this is the case of so called zero-day attacks that have no prior information; ii) considering the case of streamed data analysis, which is necessary when dealing with extremely time-variant streams, whereby the FS process should be repeated across time (e.g. by using a mobile time window), so as to periodically update the resulting dataset with the freshest features; iii) designing routines to automatically manage the best FS strategies to be applied in accordance to specific criteria (e.g. accuracy target, latency needs, etc.). Our investigation goes into the direction of the 6G paradigm that, according to most network scientists, will be characterized by intelligent resource management, smart adjustments, and automatic service provisioning.