{'fraction_non_alphanumeric': 0.03146320461018746, 'fraction_numerical': 0.00986951049877459, 'mean_word_length': 4.6759398496240605, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
S-Shaped vs. V-Shaped Transfer Functions for Antlion Optimization Algorithm in Feature Selection Problems

Feature selection is an important preprocessing step for classification problems. It deals with selecting near optimal features in the original dataset. Feature selection is an NP-hard problem, so meta-heuristics can be more efficient than exact methods. In this work, Ant Lion Optimizer (ALO), which is a recent metaheuristic algorithm, is employed as a wrapper feature selection method. Six variants of ALO are proposed where each employ a transfer function to map a continuous search space to a discrete search space. The performance of the proposed approaches is tested on eighteen UCI datasets and compared to a number of existing approaches in the literature: Particle Swarm Optimization, Gravitational Search Algorithm, and two existing ALO-based approaches. Computational experiments show that the proposed approaches efficiently explore the feature space and select the most informative features, which help to improve the classification accuracy.


INTRODUCTION
Feature Selection (FS) plays a vital role in machine learning since it aims to reduce the data size by eliminating the irrelevant/redundant features from the original datasets [1]. The use of FS algorithms in conjunction with a classification algorithm improves the classification accuracy and/or reducing the processing time [2]. FS methods can be classified based on two main criteria; searching the feature space for the near optimal feature subset and the evaluation of the selected subsets. Evaluating the selected subsets can be classified into two different approaches; Filter and Wrapper. A filter approach usually evaluates the subset depending on the data itself, whereas a wrapper approach uses an external learning algorithm (mostly machine learning technique) to evaluate the selected features.
Searching a feature space to find the best combination of features is an NP-hard problem [1]. Therefore, using brute-force search techniques are impractical with FS problems especially for the medium and large-scale datasets. As an alternative, heuristic methods can be used to find the near-optimal subset faster than brute-force methods [3]. Evolutionary Computation (EC) are population-based meta-heuristic algorithms with a global search capability [4]. Most of such methods are nature-inspired and mimic the behavior (social and biological) of animals, insects or birds like whales, bees, ants, antlions, bats, etc. [5]. In EC, a population of individuals (solutions) interact to obtain the optimal solution [4]. EC have been widely employed to tackle FS problems in the literature [6]. For instance, there are feature selection methods based on Genetic Algorithm (GA) [7,8], Particle Swarm Optimization (PSO) [9], Ant Colony Optimization (ACO) [10,11], Differential Evolution (DE) [12], and Artificial Bee Colony (ABC) According to Mirjalili and Lewis in [25], a transfer function is an important part in the binary versions of the metaheuristics. It significantly impacts the local optima avoidance and the balance between exploration and exploitation. In this paper, six transfer functions, proposed in [25], are applied to the ALO algorithm. A total of three s-shaped and three v-shaped are employed as the first attempt in the literature to find a suitable transfer function for the ALO algorithm. A wrapper model that uses K-Nearest Neighborhood (KNN) classifier is adopted as an evaluation criterion in this work. The results are compared with the two variants of basic ALO algorithm, PSO [26], and GSA [27] for verification.
The rest of this paper is organized as follows: The basic ALO algorithm is presented in Section 2. Section 3 presents the details of the proposed approaches. In Section 4, the experimental results are presented and analyzed. Finally, conclusions and future work are given in Section 5,

BINARY ALO ALGORITHM
ALO is a recent EC algorithm proposed by Mirjalili [19]. ALO algorithm mimics the interaction between the antlion insects and ants in the hunting process. In nature, an antlion digs a trap with a cone shape. The size of the trap is directly proportional to the hunger level of an antlion. Then, the antlion hides underneath the bottom of the trap waiting for an ant, which moves randomly around the trap, to fall down. Once the antlion realizes that there is an ant in the trap, it catches it. Based on this brief description of the antlion hunting process, the following items can be formulated as a set of conditions for the overall process [19]:


The ants move in a random walk in the search space. The random walk of the ants at each iteration of the algorithm is simulated as in Eq. (1) where cumsum represents the cumulative sum, n is the max iteration, t is the iteration and ( ) is a stochastic function that takes value (1) if a random number is less than 0.5 and 0 otherwise.


The traps of antlions affect the moves of the ants in the search space. Eq. 2 and Eq. 3 model this assumption: where and are two vectors that contain the minimum and maximum of all variables in t-th iteration, and are the minimum and maximum i-th ant and represents the position of the j-th antlion at the t-th iteration.


The largest trap belongs to the fittest antlion. Thus, catching an ant by an antlion is proportional to the fitness of that antlion (i.e., the antlion with the higher fitness has a higher chance to catch an ant). To model this assumption, a selection mechanism based on a roulette wheel operator is used.  To model sliding ants towards antlions, the radius of random walks of the ant is decreased adaptively using Eq. (4) and Eq.
where is a ration that controls the exploration/exploitation rate in ALO algorithm by limiting the random walk range of the ants and preys. The parameter in the above equations is defined in Eq. 6.
= 10 (6) where is the current iteration, is the max iteration, and is a constant that can adjust the accuracy level of the exploitation. is defined based on the current iteration ( = 2 when > 0.1 , = 3 when > 0.5 , = 4 when > 0.75 , = 5 when > 0.9 , and = 6 when > 0.95 ).


If an ant is caught and pulled under the sand by the antlion, then it becomes fitter than its corresponding antlion. Then, the antlion updates its position to the latest caught prey and builds a trap to improve its chance of catching another prey after each hunt. Eq. (7) models this process: where represents the current iteration, , represent the position of the j-th antlion and the i-th ant at the t-th iteration.
The antlion with the higher fitness in each iteration is considered as the elite. The elite antlion (E) and the selected antlion by using the selection mechanism (S) guide the random walk of an ant (RW1 and RW2 respectively). Since these two solutions are continuous, they must be converted to a binary version to suit the feature selection problem. The conversion is performed by applying different transfer functions that belong to two families (s-shaped and vshaped) [25]. The transfer function defines the probability of updating the binary solution's elements from 0 to 1 and vice versa. In S-shaped functions, the solution is updated based on Eq. (8) where ( + 1) is the i-th element in the solution at dimension calculated as RW1 -E or RW2 -S, rand is a random number drawn from uniform distribution ∈ [0,1].
on the position updating using a v-shaped transfer function should be done by Eq. (9) In this work, we are interested to study the impacts of different transfer functions on the performance of ALO. As such, six transfer functions proposed by Mirjalili and Lewis [25] are employed and replaced by the two transfer functions used in [6]. Table 1 shows the mathematical formulation of the transfer functions used in this paper and Fig. 2 shows these two families of transfer functions.

BINARY ALO FOR FEATURE SELECTION
In FS problems, a solution is represented as an -sized binary vector, where is the total number of features in a dataset. The complexity of generating all possible feature combinations would be 2 where a brute-force search becomes impractical. Metaheuristics are more reliable techniques for such problems. ALO is one of the metaheuristic that show a good performance in searching the feature space for the best feature subset. Each feature subset is evaluated according to two criteria; number of selected features in addition to the classification accuracy obtained when using those features. The fitness function that takes into consideration those two criteria is modeled in Eq. (10).
where (D) represents the classification error rate of a given classier (the K-Nearest Neighbor (KNN) classifier [28] is used here). | | is the number of selected features, | | is the total number of features in the dataset, and ∈ [1,0], = (1 − ) are two parameters corresponding to the importance of classification quality and subset length as per the recommendations in [6]. Algorithm 1 shows the pseudocode of the proposed approach. The ALO algorithm starts by generating two random populations for the antlions and ants. All individuals in the populations are evaluated and the best antlion is marked as the elite. Until a stopping criterion satisfied, each ant in the population updates its position with respect to either the elite solution or the solution selected by the roulette wheel. Two continuous solutions (RW1 and RW2) are generated for each ant. Hence, a transfer function is used to convert them to a binary format by defining a probability of updating the binary solution's elements from 0 to 1 and vice versa. Finally, the position of ant is updated by performing crossover between RW1 and RW2. If the fitness of the ant becomes better than that for the corresponding antlion, then the antlion's position is updated based on it.
In this study, six transfer functionsare embedded in the ALO. The three approaches that use s-shaped transfer functions are named ALO-S1, ALO-S2, and ALO-S3, while the three approaches named ALO-V1, ALO-V2, and ALO-V3 use v-shaped transfer functions.

EXPERIMENTAL RESULTS AND DISCUSSION
In this paper, a wrapper feature selection method based on BLAO is proposed. KNN is chosen as the classifier, and a Euclidean distance matrix is employed to evaluate the algorithm. Each algorithm is run 20 times with a random seed on an Intel Core i5 machine, 2.2 GHz CPU and 4 GB of RAM. Eighteen well-known benchmark datasets from the UCI data repository [29] are used to assess the performance of the proposed approaches. The details of the datasets including number of attributes and instances are shown in Table 1. The proposed methods are compared PSO [26], GSA [27], and two basic ALO algorithms (coded as bALO1 [6] and bALO2 [6]). Note that, bALO1 uses an s-shaped transfer function and bALO2 uses a v-shaped transfer function. The initial parameters and other experimental setup are presented in Table 2.  [30] 20 All algorithms are compared based on three criteria: classification accuracy, average number of selected features and the average computational time.
Inspecting Table 3, its evident that the approaches using v-shaped transfer functions are better than those equipped with s-shaped transfer functions in terms of classification accuracy (denoted in bold). This enhancement in performance may be interpreted by the abrupt switching between 0 and 1 in case of using v-shaped function that emphasize the explorative behavior of an algorithm. In addition, it may be seen that ALO-based approaches are better than GSA and PSO over all datasets. It is worth mentioning that ALO-V3 approach provides the best results on 11 datasets and outperforms GSA and PSO on all datasets, while the other proposed approaches in this work show competitive results on the majority of case studies. Table 3 also shows that ALO-based approaches perform better than PSO and GSA approaches. The reason is that ALO algorithm contains only one parameter (I) that controls the balance between exploration and exploitation. This parameter enforces a high level of exploration at the beginning of the searching process while more exploitation is imposed at the end of the optimization process. This is a bonus in avoiding local solutions. Another reason of ALO's superiority is its ability to search the feature space for the most informative features. Each ant is updated based on the best solution so far (a mechanism to highlight exploitation), and based on a solution that selected by the roulette wheel (an operator to support exploration). This gives more chances to the weak solutions to be involved in the searching process with the hope to find promising areas in the search space. Table 5 reports the computational time to obtain near optimal feature subset. Note that we re-implemented the PSO, GSA, bALO1 and bALO2 algorithms in Matlab and used the same parameter settings on all datasets to provide a fair comparison. It can be seen that ALO-V2 approach has the lowest computation time over 14 datasets. Overall ALO based approaches are better than GSA and PSO approaches in terms of computational time. This again proves the superiority of ALO algorithm on the FS problem. Table 4 includes the average number of selected features obtained from different algorithms when solving the case studies. It is observed that ALO-V3 shows better performance than other approaches on the majority of the datasets in terms of the number of selected attributes. It obtains the minimal number of selected attributes in eight datasets, while there is no other approach that could outperform other approaches in more than four datasets. The same observation can be made when analyzing the average selection ratio and classification accuracy for the same dataset; ALO-V3 has the best performance in both of them. The remarkable increase in the ALO's results when employing different transfer functions indicates the important role of the transfer function in improving the performance of the ALO algorithm.

CONCLUSIONS AND FUTURE WORK
In this paper, we studied the behavior of BALO with six different transfer functions dividing into two classes: s-shaped and v-shaped.
The proposed approaches are applied on the feature selection problems. In order to assess the performance of the proposed approaches, 18 well-known UCI benchmark datasets were used, and the results were compared with state-of-the-art approaches. It was shown that the binary ALO approaches equipped with vshaped transfer functions with their unique method of position update, significantly improve the performance of the original ALO in terms of avoiding local minima and results accuracy.
The simplicity and low-computational cost of the presented algorithms make them suitable to solve a wide range of practical problems such as the those studied in [31 -34]. For future studies, it is recommended to use the binary algorithm proposed in this wrork for data mining big sensory data sets to reduce the amount of data to be analyzed while building models with efficient interpretability using fewer features.