{'fraction_non_alphanumeric': 0.03279756733275413, 'fraction_numerical': 0.013538951636258326, 'mean_word_length': 5.272933696639419, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A two-step feature selection method for quranic text classification

Received Jan 25, 2019 Revised Apr 17, 2019 Accepted May 18, 2019 Feature selection is an integral phase in text classification problems. It is primarily applied in preprocessing text data prior to labeling. However, there exist some limitations with the FS techniques. The filter-based FS techniques have the drawback of lower accuracy performance while the wrapper-based techniques are highly computationally expensive to process. In this paper, a two-step FS method is presented. In the first step, chisquare (CH) filterbased technique is used to reduce the dimensionality of the feature set and then wrapper correlation-based (CFS) technique is employed in the second step to further select most relevant features from the reduced feature set. Specifically, the ultimate aim is to reduce the computational runtime while achieving high classification accuracy. Subsequently, the proposed method was applied in labeling instances of the input data (Quranic verses) using standard classifiers: naïve bayes (NB), support vector machine (SVM), decision trees (J48). The results report the proposed method achieved accuracy result of 93.6% at 4.17secs.


INTRODUCTION
With the advancements in information technology, the amount of documents being processed over the years have continually increased. This has made the fields of artificial intelligence (AI) and machine learning (ML) attract attention and unceasing developments. The increasingly demand for processing large documents within a short time necessitate the automating of documents processing. Automated text classification (ATC) [1] is the steps and techniques involve in automatically classifying textual data to predefined class/label. To do this, there is a need for machines to learn [2]. In the field of machine learning, the goal is to develop models that give computing machines the capability of learning [2]. Thereafter translates the acquired knowledge into decision making.
An essential phase in text classification task is feature selection; a dimensionality reduction method that helps to reduce the complexity of dimensionality usually associated with text [2]. In text, there is presence of large feature space which often results in high level of dimensionality. This often occurs when the feature set consists of both relevant, irrelevant, as well as redundant features (or attributes) which could lead to misclassification by the classifiers (overfitting) [2]. Thus, to ensure the optimization of the classification algorithms" performance, feature selection methods and techniques can be applied in preprocessing the textual data prior to the actual labeling tasks. Quran is a unique text and a good source of information with about 78,000 words. These words are very rich in text and systematically arranged by experts into various sections, groups [1]. With such great importance and characteristics, features

731
(or keywords) could be extracted from the divine text in order to help improve the literacy level of its readers and researchers. In addition, there are multiple sources of the Quranic text available from the extensive academic works of the religious scholars. From such sources are the Quran translations (translated from Arabic to almost all languages), as well as the Quran commentaries. However, experimenting these sources independently has its setbacks. For example, the Holy Quran translation as a source may not be sufficient for the purpose of analyzing Quranic verses for the labeling task. Thus, there is a need to combine the sources (otherwise termed group-based) while extracting features for the classification task. The field of Holy Quran study has witnessed a quite number of research works. These include: text classification applications of the Holy Quran [1][2][3][4][5][6]; ontology-based applications [7][8][9][10]; digitized Holy Quran applications [11][12][13][14]. Furthermore, from among the techniques that have been widely applied to text classification problems, include the Bayes probabilistic approach [15], decision trees [16], neural networks [17], support vector machines [18], and k-nearest neighbor [19]. In addition, the research work in [20] is based on classifying sonar targets using information gain FS algorithm for attribute evaluation. The experimental work focused on training networks for the purpose of discriminating between the sonar signals.
The experimental results showed IG attribute evaluation significantly improved the classification task. A Genetic algorithm wrapper-based feature selection method was proposed in [21] for classifying hyperspectral images using SVM classifier. The feature selection process involves three steps: creating the training and testing sets using ENVI software; setting up required parameters; running the model. The FS algorithm was used to optimize the kernel parameters and feature subsets. [22] introduced in their work a feature selection method using support vector machine to find dependency between the attributes of high dimensional data extracted from UCI data repository and then decide the appropriate class attributes values. The results showed that the FS method had promising results. In addition, from other research works in feature selection include but not limited to [23][24]. Due to the limitations found with the available FS techniques such as high computational cost (as associated with wrapper-based FS techniques) and lower accuracy performance (as associated with filter-based techniques), the study proposes a two-step FS method. The study aimed at reducing the computational complexity while achieving high classification accuracy results.
Hybrid approach to feature selection has been successfully experimented in classification problems [25][26][27][28][29][30]. The proposed method is a combination of chisquare (CH) filter-based and wrapper-based CFS algorithms. The two-step CH-CFS method will be applied in labeling the verses of the Quranic datasets using naïve bayes (NB), support vector machine (SVM), and decision trees (J48) classification algorithms. The input verses are classified into three predefined labels: "iman, ibadah, and akhlak". These class labels are from the most fundamental aspects of Islam [1,3].

METHODS AND MATERIALS
The experimental design as shown in Figure 1 consists of five steps. The input data are Quranic verses gathered from the combined sources of Holy Quran translation and tafsir. The resulting combined text data is otherwise termed "grouped-data". The experimental phases include: data gathering, feature generation, feature selection, classification, and output results.

Data Gathering
The experimental datasets as tabulated in Table 1 comprises of 451 instances (quranic verses) made up of 286 verses from chapter two (Surah al-Baqarah) and 165 verses from chapter six (Surah al-Anaam) of the Holy Quran. As could be seen from the class weight distribution, the "iman" class has the most class members (input verses).

Feature Generation
Features are first extracted from the Quranic texts. To do this, the study employed standard StringToWordVector filter tool [1]. Furthermore, TF-IDF weighting method is applied to access and measure the degree of relevance of the extracted features. Term frequency ( ) as given in (1) is an important preprocessing method used in measuring the frequencies of words in textual data [1][2].
where is denoted with * +.

Figure 1. Proposed framework
Also, inverse-document frequency (IDF) helps to measure the relevancy of a given word. The method is given as: Generally, TF-IDF is given as:

Feature Selection
Dimensionality reduction methods such as feature selection are mostly employed to reduce the curse of dimensionality. This problem is often associated with textual data. The presence of high dimensionality may influence negatively classifier"s decision making. Feature selection method can be applied either by ranking the features/attributes or through subset selection approach [2]. The ranking features approach ranks features according to a certain criterion of the feature selection algorithms with the top k features selected. On the other hand, the subset selection approach selects a minimum subset of features without learning performance deterioration [2].
In this paper, the ranking features approach was experimented using chisquare (CH) filter-based algorithm while for the CFS wrapper-based algorithm was applied for the subset selection approach. Filter method is less computationally expensive in comparison with the wrapper method. The filter method selects features independent of the classifiers. This makes the method simple, fast, and less expensive to run. On the other hand, wrapper method utilizes the performance of the classification algorithms to evaluate and select the feature subsets. This makes wrapper FS method perform better but with high computational cost.

733
Chisquare filter algorithm is used as a test of independence to access the independence of the class label of a particular feature. Given a feature with different values and classes, chisquare feature score can be defined as: Where n_ij is the number of samples with the i^th feature value. CFS wrapper-based method finds the subsets of features that are individually highly correlated with the class but have low inter-correlation [2]. The method can be calculated using: The experimental workflow of the proposed methodology follows the following steps: Step 1: Input data (quranic verse) Step 2: Generate features from input data using StringToWordVector and TF-IDF Step 3: Feature preprocessing using CH feature selection algorithm Step 4: Implement features from (step 3) with the classifiers Step 5: Select features from (step 2) using CFS algorithm Step 6: Implement selected features from (step 5) with the classifiers Step 7: Load the generated features from (step 2) Step 8: Iterate (step 3) Step 9: Apply CFS algorithm on selected features from (step 8) Step 10: Implement resulting features from CH-CFS with the classifiers Step 11: Evaluate results

Classification (Labeling)
For the labeling task, the experimental work implemented three of the conventional classification algorithms: NB, SVM, and J48 classifiers. These algorithms are widely applied to several classification problems [2]. For data partitioning, we employed the standard 10-fold cross validation method. NB classifier is a simple probabilistic model based on the Bayes rule [2]. Given a class , the probabilty of a particular document to belong to is given as: SVM algorithm is typically used for learning classification, regression, or ranking function. The algorithm works by searching a seperating hyperplane to seperate between samples with a maximal margin [2]. The equation for hyperplane is: In decision tree classification algorithm, each node specifies a test to be performed on a single attribute [1]. The goal is to create a model that predicts the value of a target variable based on several input variables. The data generally takes the form: Finally, the study validated the experimental results using accuracy performance. Classification accuracy is one of the widely used performance metrics in text classification problems [2]. Given a confusion matrix, the accuracy metric is calculated as:

EXPERIMENTAL RESULTS AND ANALYSIS
The FS algorithms experimented on the Quranic datasets produced mixed results as shown in Tables 2-4  Working with the entire generated features (without feature selection) could affect the classifiers" performance as shown in the experimental results. Applying all the features directly produced the least accuracy result of 64.3% with J48 algorithm on the group-based QTrans+Taf dataset. The result was obtained as a result of high dimensionality in the text data. To optimize the classification results, feature selection techniques were applied. The curse of dimensionality could be solved by applying FS algorithms on the text data prior to classification. Consistently, the feature selection algorithms obtained above 80% accuracy results.
The filter-based CH and the wrapper-based CFS algorithms obtained with SVM classifier 92.8% at 1.53secs and 93.6% at 119.6secs accuracy results respectively. Analysis of these results showed the limitations earlier identified with the existing FS methods. techniques are less efficient and relatively achieve lower accuracy results. However, the wrapper-based techniques are computationally expensive to work on as seen in the results. It took CFS algorithm a high computational runtime of 119.6secs to select features from the feature set. Consequently, the proposed CH-CFS algorithm achieved the overall highest accuracy performance of 93.6% at a very less computational runtime of 4.17secs with SVM classifier.

CONCLUSION
Feature selection process as experimented in this study has proven to be an integral phase in text classification tasks. The study identified some limitations with the existing FS techniques. To address these setbacks, the study proposed a hybridized FS method. The proposed method is a two-step combination of filter-based chisquare and wrapper-based CFS algorithms.
The specific goal of the study is to apply FS algorithms in automating the labeling of Quranic verses. The set target is to achieve with the proposed CH-CFS higher classification accuracy performance at lower computational runtime. The proposed technique achieved the overall accuracy result of 93.6% at 4.17secs in comparison with the wrapper-based CFS algorithm which achieved the same accuracy result but at a high computational runtime of 119.6secs. In future work, the study will focus on extending the proposed hybrid CH-CFS algorithm to other classification problems.