{'fraction_non_alphanumeric': 0.023491027732463296, 'fraction_numerical': 0.008809135399673736, 'mean_word_length': 4.935708752904725, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Usage of Particle Swarm Optimization to Improve the Performance of Supervised Classifiers

Representing the data appropriately will have a significant effect on the outcome produced by the classifier. Transforming the feature will help to represent the data points in a more suitable way for the classifier. Particle swarm optimization belongs to the Swarm Optimization techniques category and it is generally used for solving numerical optimization problems, weight updation in neural networks, and feature selection. This research work proposes a Particle swarm optimization-based transformation technique for increasing the classification metrics of popular classification algorithms namely K-Nearest Neighbor, Stochastic Gradient Descent Classifier, Decision trees, and linear discriminant analysis classifier. Experiments are conducted using the SONAR dataset and the highest accuracy of 82.69% is attained for Stochastic Gradient Descent classifier when Particle swarm optimization is used as a transformation technique


INTRODUCTION
There is a need for classification tasks in a variety of fields. Based on the past observations, the categorical class labels of new instances will be predicted through a supervised classification task. To choose the most appropriate algorithm to solve the problem it is always recommended to compare and try various algorithms' performances. Adequate classification performance cannot be achieved by many classification/learning algorithms due to the poor quality data space representation. When facing difficult problems, the solution for this deficiency is the usage data transformation technique. To identify the implicit correlation of data points, new data points will be produced for enhancing the suitability of data points for the classification task. The constructed feature usually acts as function of the original low-level features [1][2].
Nowadays evolutionary techniques are widely used in various fields [3][4]. Swarm Intelligence (SI) algorithms belong to a sub-category of evolutionary techniques and they are based on the behavior of a group of animals [5]. Some of the SI algorithms include Particle Swarm Optimization (PSO), Dragonfly algorithm, Bat algorithm, Elephant herding optimization, etc. PSO is easier to implement, less expensive, can converge more quickly, and having very few parameters comparatively. In many complex problems, PSO has been successfully implemented as a powerful search technique. In this research work, PSO will be tested as a transformation technique on four different classifiers namely, K-Nearest Neighbor (KNN) classifier, Stochastic Gradient Decent (SGD) Classifier, Decision Trees (DT), and Linear Discriminant Analysis (LDA) classifier. All the experiments are performed on the SONAR dataset in this research work. A brief introduction to SONAR dataset is given below: For building this dataset, a 5 feet long metal cylinder is used to represent the minerals and a rock of the same length is used. A wide-band frequency modulated signal is targeted on the metal cylinder and rocks. At various angles and distances, 208 times the experiment is repeated. Sample signals received from minerals and rocks are shown in Figure 1. Out of 208 instances, 111 belong to the minerals class and 97 belong to the rock class. In each instance, 60 echo values are recorded and they differ between 0 and 1. It will be a tedious task for manual inspection and sometimes it may lead to erroneous results. Hence automated classification of SONAR Dataset is proposed to categorize the SONAR features into two classes: Minerals and Rocks. This type of automated classification will be very much useful in the mining industry.
The organization of the remaining paper is given below: The basics of PSO and classifiers used will be explained in the next section; implementation methodology will be elaborated in the second section; results will be presented along with discussion in the third section and the conclusion is presented in the last section.

Particle Swarm Optimization
PSO technique tries to mimic the flying mechanism of bird flocks [6][7]. PSO is one of the iterative optimization algorithms and the velocity & position of birds (particles) are updated using equations (1) & (2). Here v i and x i denotes the velocity & position of i th particle respectively; t represents the current iteration; ( ) specifies the individual finest position of the i th particle; gbest signifies the over-all best All the experiments are performed on the SONAR dataset in this research work. A brief introduction to SONAR dataset is given below: For building this dataset, a 5 feet long metal cylinder is used to represent the minerals and a rock of the same length is used. A wide-band frequency modulated signal is targeted on the metal cylinder and rocks. At various angles and distances, 208 times the experiment is repeated. Sample signals received from minerals and rocks are shown in Figure 1. Out of 208 instances, 111 belong to the minerals class and 97 belong to the rock class. In each instance, 60 echo values are recorded and they differ between 0 and 1. It will be a tedious task for manual inspection and sometimes it may lead to erroneous results. Hence automated classification of SONAR Dataset is proposed to categorize the SONAR features into two classes: Minerals and Rocks. This type of automated classification will be very much useful in the mining industry.
The organization of the remaining paper is given below: The basics of PSO and classifiers used will be explained in the next section; implementation methodology will be elaborated in the second section; results will be presented along with discussion in the third section and the conclusion is presented in the last section.

Particle Swarm Optimization
PSO technique tries to mimic the flying mechanism of bird flocks [6][7]. PSO is one of the iterative optimization algorithms and the velocity & position of birds (particles) are updated using equations (1) & (2).
Here v i and x i denotes the velocity & position of i th particle respectively; t represents the current iteration; ( ) specifies the individual finest position of the i th particle; gbest signifies the over-all best All the experiments are performed on the SONAR dataset in this research work. A brief introduction to SONAR dataset is given below: For building this dataset, a 5 feet long metal cylinder is used to represent the minerals and a rock of the same length is used. A wide-band frequency modulated signal is targeted on the metal cylinder and rocks. At various angles and distances, 208 times the experiment is repeated. Sample signals received from minerals and rocks are shown in Figure 1. Out of 208 instances, 111 belong to the minerals class and 97 belong to the rock class. In each instance, 60 echo values are recorded and they differ between 0 and 1. It will be a tedious task for manual inspection and sometimes it may lead to erroneous results. Hence automated classification of SONAR Dataset is proposed to categorize the SONAR features into two classes: Minerals and Rocks. This type of automated classification will be very much useful in the mining industry.
The organization of the remaining paper is given below: The basics of PSO and classifiers used will be explained in the next section; implementation methodology will be elaborated in the second section; results will be presented along with discussion in the third section and the conclusion is presented in the last section.

Particle Swarm Optimization
PSO technique tries to mimic the flying mechanism of bird flocks [6][7]. PSO is one of the iterative optimization algorithms and the velocity & position of birds (particles) are updated using equations (1) & (2).
Here v i and x i denotes the velocity & position of i th particle respectively; t represents the current iteration; ( ) specifies the individual finest position of the i th particle; gbest signifies the over-all best

KNN Classifier
KNN classifier [8] relies on a simple classification methodology based upon the distance and neighbors. The main advantage of the KNN classifier is it won't make a prior assumption of data. But the prediction time is quite high as it has to find the distance between every data point. Different distance metrics like Euclidean distance, Chebyshev distance, Manhattan distance, etc. can be used in the KNN classifier. For example, Euclidean distance between two points p and q are found using the equation,

SGD Classifier
Basically SGD [9] is an optimization function that tries to minimize the cost function by tuning the values of control parameters. It can be applied to various machine learning tasks.

DT Classifier
The decision tree [10] tries to build a tree architecture for modeling the classification or regression tasks. It will decompose the data points into sub-groups in an appropriate manner using a set of procedures.

LDA Classifier
LDA is a supervised classification method that is used to create machine learning models [11]. These models based on dimensionality reduction are used in the application, such as marketing predictive analysis and image recognition, amongst others. LDA is a simple and well-understood technique that is commonly used in classification ML models. Figure 2.represents the flowchart that depicts the methodology used in this research work. The features from the SONAR dataset will be directly given to PSO which acts as a transformation technique. The transformed data points from PSO's output are given to any one of the four supervised classification techniques namely, KNN, SGD, DT, and LDA classifier. Finally, the performance will be assessed using five popular performance metrics namely Accuracy, Error Rate, Mathews Correlation Coefficient (MCC), Precision, Recall, and F1 score.

Figure 2.Flowchart depicting the methodology used in this research work
In general, if PSO is used for feature selection and solving an optimization problem, the particles will be initialized randomly. But to act as a transformation technique, the particles are initialized with features from the SONAR dataset. Fitness function is computed as the reciprocal of Euclidean distance from target to the current particle's position. Depending upon the fitness values, global best and personal best is computed and then equations (1 & 2) are used to update the position of particles in each iteration.
The ideal values for the maximum number of iterations, target, three control parameters (w, c 1 , and c 2 ) are found using the trial and error method. For instance, Figure 3 depicts the process of finding the ideal values for PSO control parameter w. In general, if PSO is used for feature selection and solving an optimization problem, the particles will be initialized randomly. But to act as a transformation technique, the particles are initialized with features from the SONAR dataset. Fitness function is computed as the reciprocal of Euclidean distance from target to the current particle's position. Depending upon the fitness values, global best and personal best is computed and then equations (1 & 2) are used to update the position of particles in each iteration.
The ideal values for the maximum number of iterations, target, three control parameters (w, c 1 , and c 2 ) are found using the trial and error method. For instance, Figure 3 depicts the process of finding the ideal values for PSO control parameter w. In general, if PSO is used for feature selection and solving an optimization problem, the particles will be initialized randomly. But to act as a transformation technique, the particles are initialized with features from the SONAR dataset. Fitness function is computed as the reciprocal of Euclidean distance from target to the current particle's position. Depending upon the fitness values, global best and personal best is computed and then equations (1 & 2) are used to update the position of particles in each iteration.
The ideal values for the maximum number of iterations, target, three control parameters (w, c 1 , and c 2 ) are found using the trial and error method. For instance, Figure 3 depicts the process of finding the ideal values for PSO control parameter w.

3.RESULT AND DISCUSSION
The performance metrics of four different supervised classifiers with and without PSO as transformation technique for the classification of test SONAR dataset are shown in Table 1. MCC can be considered as an efficient performance metric that gives an overall picture about the classification [10] and meaningful insights can be derived through MCC comparison for various classifiers and transforms as shown in Figure 4.  From Table 1, it can be witnessed that the accuracy of four different supervised classifiers namely KNN, SGD, DT, and LDA has accuracy in the range of 73-79% when no transformation techniques are used. If PSO is used as a transformation technique, the accuracy of all the four supervised algorithms is getting increased and the highest accuracy of 82.69% is attained for the PSO-SGD

3.RESULT AND DISCUSSION
The performance metrics of four different supervised classifiers with and without PSO as transformation technique for the classification of test SONAR dataset are shown in Table 1. MCC can be considered as an efficient performance metric that gives an overall picture about the classification [10] and meaningful insights can be derived through MCC comparison for various classifiers and transforms as shown in Figure 4.  From Table 1, it can be witnessed that the accuracy of four different supervised classifiers namely KNN, SGD, DT, and LDA has accuracy in the range of 73-79% when no transformation techniques are used. If PSO is used as a transformation technique, the accuracy of all the four supervised algorithms is getting increased and the highest accuracy of 82.69% is attained for the PSO-SGD

3.RESULT AND DISCUSSION
The performance metrics of four different supervised classifiers with and without PSO as transformation technique for the classification of test SONAR dataset are shown in Table 1. MCC can be considered as an efficient performance metric that gives an overall picture about the classification [10] and meaningful insights can be derived through MCC comparison for various classifiers and transforms as shown in Figure 4.   classifier. Not only accuracy, but all the performance metrics are also moving towards the ideal value when PSO is used as a transformation technique for the above mentioned four classifiers. From Figure  4, the increase in MCC due to PSO transform can be witnessed. Remarkably around a 26% increase in MCC can be observed for PSO-KNN classifier over KNN classifier.

CONCLUSION
Performance enhancement attained by four popular supervised classification algorithms using PSO as a transformation technique is analyzed in this research work. It is observed that PSO provides good performance improvement for all the four classifiers tested but the level of improvement varies with the classifier. For instance, a 26% increase in MCC is witnessed in KNN due to PSO usage and only an 8% increase in MCC is witnessed in LDA due to the usage of PSO. In the future, the efficiency of PSO as a transformation technique has to be studied for other datasets and other applications.