{'fraction_non_alphanumeric': 0.03073886068809927, 'fraction_numerical': 0.016074450084602367, 'mean_word_length': 4.754462613905879, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
EHHR: an efficient evolutionary hyper-heuristic based recommender framework for short-text classifier selection

With various machine learning heuristics, it becomes difficult to choose an appropriate heuristic to classify short-text emerging from various social media sources in the form of tweets and reviews. The No Free Lunch theorem asserts that no heuristic applies to all problems indiscriminately. Regardless of their success, the available classifier recommendation algorithms only deal with numeric data. To cater to these limitations, an umbrella classifier recommender must determine the best heuristic for short-text data. This paper presents an efficient reminisce-enabled classifier recommender framework to recommend a heuristic for new short-text data classification. The proposed framework, “Efficient Evolutionary Hyper-heuristic based Recommender Framework for Short-text Classifier Selection (EHHR),” reuses the previous solutions to predict the performance of various heuristics for an unseen problem. The Hybrid Adaptive Genetic Algorithm (HAGA) in EHHR facilitates dataset-level feature optimization and performance prediction. HAGA reveals that the influential features for recommending the best short-text heuristic are the average entropy, mean length of the word string, adjective variation, verb variation II, and average hard examples. The experimental results show that HAGA is 80% more accurate when compared to the standard Genetic Algorithm (GA). Additionally, EHHR clusters datasets and rank heuristics cluster-wise. EHHR clusters 9 out of 10 problems correctly.


Introduction
Recent efforts in machine learning (ML) focus on automation of the heuristic selection process by incorporating domain independence [1]. The search techniques for selecting heuristics are generalized by making the search process independent of the considered domain. However, the heuristic approaches and other search strategies in solving real-world computational search issues are challenging in freshly-discovered and new instances of the same problem. Such problems arise from various parameters, algorithm choices, and the absence of selection criteria [2]. Hyper-heuristics approaches are generic and domain-independent to search and solve a problem set rather than a single problem. These approaches target a heuristic search space instead of a solution search space. These approaches have been extensively researched in timetabling, bin packing, data mining, and feature selection [3]. However, their application for classifier selection in the domain of short-text data has not been paid much attention. Short-text is an important source of data from online platforms like microblogs, e-commerce systems, and social networks, in the form of comments, tweets, and reviews [4,5]. It is called short-text as fewer characters are used to provide what people think, such as a tweet can have a maximum of 280 characters. Various industries analyze the social media-generated comments and user feedback by using ML approaches to identify problems, elicit requirements [6], provide interest-based recommendations [7], improve their products and services for the end users [8,9].
Varying backgrounds, language usage, and writing styles of social media users instigate the lexical, syntactic, and semantic (language morphology) perplexity of shorttext, thus contributing to the diversity of such data [10]. In addition, since most words appear only once and there are not enough contexts to clarify the meaning of ambiguous words, short-text experiences severe data sparsity problems compared to long text [11,12]. The key problem with short-text classification is the brevity of the short-text and the sparsity of the feature space [13]. Various statistical, Machine Learning (ML), and Deep Learning (DL)-based heuristics have been developed to classify task-specific short-text data [14][15][16][17][18][19][20][21]. In accordance with the ''No Free Lunch theory [22,23],'' there is no single heuristic that works best for all short-text classification problems. Brute force search of the best classifier for such big short-text data is expensive yet impractical in terms of time and computational resource utilization [24]. Additionally, selecting an acceptable heuristic for addressing a short-text classification problem is not simple since the problem's features are typically not well understood in advance [25]. In this context, how to automatically recommend the best heuristic is a major concern for classifying a short-text problem at hand, especially for non-experts [26,27]. Although, existing research focused on classifier recommendations. However, they only deal with numeric or categorical data [1,28,29]. Some of the meta-learningbased techniques [30,31] utilize the human expert opinion for recommending the best classifier on numeric data. Motivated by these concerns, it is inevitable to determine the best heuristic for classifying a short-text dataset and conserving the computational resources, especially for high volume, sparse, and diverse short-text datasets.
This paper aims to develop a hyper-heuristic framework (see Fig.1) to recommend the best short-text classifier. Figure 1 depicts the working of the classifier recommender. Classifiers' from the classifiers pools are executed on the training datasets to achieve their relative performances and rank and reminisce them. The hyper-heuristic uses datasetlevel features of the test dataset to predict performance and recommend the best classifier based on the reminisced classifier ranking. The framework is intended to make an intelligent choice of a machine learning classifier less timeconsuming by reusing the performances of heuristics on previously solved problems. The EHHR employs an evolutionary algorithm called Hybrid Adaptive Genetic Algorithm (HAGA), which serves as a hyper-heuristic. HAGA contributes to determining the most influential dataset-level features and performance prediction. Based on the predicted performance, the ranks of classifiers are determined through a clustering algorithm.

Contributions of the study
The contribution of this study is three-fold. It recommends the best-suited classifier for short-text data from a heuristic space (i.e., a pool of classifiers). It also determines the best dataset-level features playing an important role in choosing the best classifier. It minimizes the need for a stochastic search of classifiers for a dataset and automates the choice of the best classifier. The following are the major contributions of this paper.
• A novel Hyper-Heuristic Classifier Recommender (EHHR) to automatically recommend the best heuristic for the short-text classification is proposed. • A Hybrid Adaptive Genetic Algorithm (HAGA) for determining the most influential dataset-level features in EHHR, is proposed. • A new framework is presented to investigate the performance of heuristics for the large volume, varying sized, balanced, and imbalanced data. • A reminisce-enabled model has been devised to minimize the need for the stochastic search for the most suitable classifier for an unseen dataset.
The rest of the paper is organized as follows. Section 2 presents the related work to highlight the gaps in the previous studies. The modules and operations of the proposed technique have been discussed in detail in Sect. 3. In the Sect. 4, we provide the details of the experimental setup. Section 5 contains detailed results and analysis. Section 6 concludes the study.

Related work
Heuristic techniques have been widely applied to perform data classification tasks [32,33]. A heuristic for one dataset may not be equally effective for another dataset [34]. Moreover, they are expensive due to their problem-specific nature [2] and lose their generality due to customization and parameter tuning. Different studies focused on recommending an appropriate heuristic (i.e., an appropriate classifier), which is answered in the context of the numeric form of data, as discussed ahead. Pise and Kulkarni [35] used simple, information-theoretic, and statistical meta-features for algorithm selection.
To recommend the best algorithm, the suggested technique used the K-Nearest Neighbor (KNN) algorithm and the classifier's accuracy as the recommendation measure. The experiment indicates that calculated accuracies match actual accuracies for over 90% of the benchmark datasets used. However, the approach suffers from computation overload due to the inefficient selection of meta-features, which results in calculating all meta-features. The classifier selection for real-time intrusion detection [36] necessitates a tedious re-evaluation process resulting from changing classifiers.
The study [28] uses real-coding evolutionary algorithm to present Automatic Machine Learning (Auto-ML) based solution for automatic classifier recommendation. Statistical and structural information-based method of feature vector generation using KNN [37] only works with binary data sets and cannot discriminate between the target and other characteristics. Furthermore, calculating itemset frequency for high-dimensional data takes more time and space. Wang's classifier recommendation [38] technique is based on extracting attribute correlations from the data set's structure but results in high computational costs for itemset generation. Link prediction through DAR (Data and Algorithm Relationship) Network [39] suffers from the choice of optimal value for k (which remained fixed for all datasets). CB-MLR [31] and AMD [30] need human experts' involvement in defining application-specific goals and for empirical evaluation of classifiers' performance based on quality meta-metrics, respectively.
The EML (i.e., Ensemble of ML-KNN ) is a two-layerrecommendation technique that uses diverse meta-features and KNN to measure similarity between predicted and archived problems. However, setting an appropriate K value requires expert knowledge [26]. Li et al. [29] proposed a meta-learning-based technique for solving engineering problems in manufacturing systems. The framework proposed dataset, classifier algorithm, and recommendation modules to suggest several algorithms for a dataset. However, the technique heavily relied on an accurate selection of hit rate and accuracy to avoid misleading recommendations of an algorithm. Corrales et al. [40] proposed an algorithm recommender for regression and classification problems using case-based reasoning. The limitation of the system lies in that the efficiency declines when there is an increase in search time for similar solved cases.
To make an appropriate classifier recommendation, an important aspect is to achieve it through domain independence of the problem, which is mainly realized using hyper-heuristic approaches. Evolutionary techniques have demonstrated their significance as hyper-heuristic techniques for problem areas such as bin-packing [41], image segmentation [42], timetabling [43]. Hyper-heuristic (HH) Fig. 1 Handling of classifier selection problem through Hyper-heuristic. C ri is classifier's rank, f i are dataset-level features. P i is the performance of classifier C i . C r3 is the best-predicted classifier for testing the dataset Cluster Computing has been used in a variety of fields, especially with classification problems [44]. The major issue with these approaches is that they only apply to specific problems and do not store and reuse the previously solved problems to guide the search of heuristics for the unseen instances. To overcome this issue, the Deja Vu framework maintains an Acquired Knowledge Module(AKM) to store problem definitions, fitness functions, and best-performing heuristics for each problem [1]. The framework solves the new problem based on similarity to an existing problem.
Though the framework has been tested with various classification problems, it does not apply to text categorization problems due to text data's unstructured nature. Hence, the requirement of a hyper-heuristic to tackle the classifier recommendation of short-text for balanced, imbalanced, and voluminous datasets arises.
The above-mentioned techniques work with numeric and categorical data (i.e., see Table 13in Appendix A). The meta-learning techniques lack the optimization of features, resulting in computational and memory overhead. Furthermore, some of them introduce a high reliance on human expertise. To the best of our knowledge, no classifier recommendation has been presented to date to cater to the needs of short-text data.

Proposed technique
The proposed Evolutionary Hyper-Heuristic Classifier Recommender (EHHR) framework intends to determine the ranking of the various heuristics based on their performance in classifying short-text data. This framework comprises four segments: Feature Extraction Module (FEM), Heuristic Evaluation Module, Evolutionary Module, and Memory Module. These segments contain dataset features, a list of heuristics with their parameter settings, and heuristics' performance for each dataset. Figure 2 shows the functioning and interaction of the modules of the EHHR framework. Data originating from various data sources is preprocessed and then input to the FEM, which calculates the 18 features for each dataset. The heuristic evaluation module of the EHHR framework computes the performance of multiple heuristics for each dataset. The heuristic space comprises a set of classifiers which are in the Classifier Pool in Fig. 1. The evolutionary module of EHHR optimizes and predicts the most influential dataset features and the average classifier performance. The heuristic performances help to group datasets into clusters and predict their ranking. The memory module stores the clusters' information and heuristics' ranking along with their best parameter values. This information is reminisced to recommend the best classifier for a new problem.

Feature extraction module (FEM)
FEM facilitates the computation of the linguistic complexity of a dataset. It computes different dataset-level features for the problem domain N to compute the linguistic complexity. These features belong to lexical, syntactic, or word length categories (reference Table 1). Lexical measures help the framework evaluate the lexicon's worth in text and use lexical sophistication and lexical variation measures. Syntactic measures are computed through syntactic complexity indices, and the readability scores quantify the complexity of short-text in terms of reading ease. It produces a feature set matrix (NxD dimensions) as shown in algorithm 1, where D is datasetlevel features, and N represents a dataset in the problem space. Explanation of the features, taken from the literature [10,[45][46][47][48][49][50] is provided in Appendix B with respect to types and token.

Proposed features
Along with the features presented in table 1, we also developed the following features for our framework.
2. Average Entropy of frequently occurring words: The average entropy of the most frequent words is computed for each dataset. The average entropy measures the informational value contributed by the frequently occurring words in the dataset according to the Eq. 2. w j is a word type in W, n is the total number of words in the corpus, also known as vocabulary size, while probðw j Þ is the probability of w j in the corpus. probðw j Þ is the ratio of the frequency of w j to the sum of all type frequencies within n. f is the number of frequent words.
3. Average TF-IDF: Average TF-IDF is the mean of TF-IDF values of the most frequent words. The average values help to quantify the central tendency of the most relevant words in the short-text. If Y i represents the TF-IDF value of a word x i X, and f is the number of top frequent words. Then the average TF-IDF is calculated as presented in Eq. 3:

Heuristic evaluation module
The heuristic evaluation module unmasks the actual performance of heuristics (i.e., see Fig. 3) for problems being considered. The heuristic space of the proposed framework comprises a pool of state-of-the-art classification algorithms. Each heuristic h from the heuristic search space H is trained on (the embeddings for) each dataset n from the set of datasets N, and its actual performance is calculated as shown in algorithm 1 (lines 8-13). The proposed algorithm uses macro f1-score as the performance measure. Generally, the recommendation frameworks use accuracy as a performance measure. However, the proposed framework used macro f1-score, which is a good predictor for varying sized and highly skewed (i.e., imbalanced) datasets. Algorithm 1 computes the heuristic evaluation vector (Nx1 dimensions). The average macro f1-score of all heuristics in H for each dataset is computed and recorded. The heuristic evaluation is only applied during the training phase of the framework and is computationally expensive. However, these computations serve as the basis for the reminiscence (i.e., recalling the classifier's past performances) and reusability of the low-level heuristic's previously-stored performances to solve new problems. Furthermore, the module saves the best parameter values for all heuristics. For a dataset n N, the f1-score (macroF1 nh i ) is computed Fig. 3 The heuristic evaluation process in EHHR resulting in computation of actual performance in terms of average macro-F1 for each dataset n N

Cluster Computing
for each h i . The Eq. 4 presents the calculation of the average macro f1-score for all heuristics for each dataset.
In Eq. 4, h i refers to heuristic and n is the dataset. Variable i is used to iterate through 1 to z, where z is the total number of heuristics in the heuristic space.

Evolutionary module
In the evolutionary module, the proposed Hybrid Adaptive Evolutionary Algorithm (HAGA) is trained to predict the average performance for the given data. HAGA is an improvement to the general GA applied in a hyperheuristic setting. For a population of l chromosomes, the selection process of HAGA is the hybridization of tournament selection with elitism. To cater to the problem of a less diverse population and premature convergence arising due to elitism, some chromosomes are randomly selected for the new population. Furthermore, the adaptive mutation and random crossover in HAGA encourage the exploration of global optimum and population diversity. In the training phase, HAGA takes the extracted features (explained in Section Feature Extraction Module (FEM)) as input data and the actual average performances of the classifiers for each data as true values. The average macro f1-score (computed in the heuristic evaluation module) is the true label for training the HAGA.
The chromosomes of the HAGA contain the weights for the input features. The weights are multiplied by the features and summed up. The output value presents the predicted average performance. The fitness function of HAGA is a minimization function that evolves the weights in order to minimize the loss. The difference between the predicted and actual performance is taken as the loss. The actual performance is the average performance of each dataset w.r.t. macro f1-score. Equation 5 computes the average loss for all datasets.
The detailed pseudo-code for the HAGA is given in Algorithm 2. The convergence (very low update in loss) of the HAGA is considered the termination criteria. The key differences between the basic GA and the proposed HAGA are given below: 1. Mutation value is adaptive. Mutation rate changes after every x% of iterations instead of fixed mutation value. 2. Hybrid selection strategy has been applied for selecting chromosomes for the next generation. 3. The crossover point is not fixed; we select random points for each crossover to ensure diversity in the population.
The output of the HAGA is the weight vector that best minimizes the loss. In the testing phase, for any given dataset, the features vector (N Â 1) is multiplied by the weight vector. It returns the predicted average performance Cluster Computing of that dataset. For any given dataset, the average performance will be predicted using HAGA rather than the training of the machine learning classifiers.

Memory module
The memory module of EHHR stores the results produced by all modules and clustering algorithms to reminisce them for new datasets. The EHHR uses a fuzzy C-means clustering algorithm to cluster datasets. The primary objective of the clustering process is to decrease the dissimilarity between the data allocated within the same cluster [52]. Clustering is a technique that can group classifiers with similar performances within the same cluster to minimize the complexity of predicting one or more classifiers to a user. Fuzzy c-means (FCM) is a type of clustering technique in which it is possible to cluster values that lie in two or more clusters with similar degrees of membership without any difficulty [53]. Hence, this module groups the datasets into clusters based on their performances.
The memory module stores the obtained clusters of datasets based on performance, with best-performing heuristics and their parameter settings obtained during the training phase. Each cluster provides the ranking of lowlevel heuristics.

Handling an unseen problem
EHHR helps determine the best classifier for a new problem without stochastically running all heuristics. First, the features presented in Section Feature Extraction Module (FEM) are extracted for the new dataset. These features are used to predict the performance with the help of HAGA. The cluster is assigned to the dataset based on the predicted performance. The best algorithm for the chosen cluster is applied to the new problem. The memory module is updated with the extracted feature values and the chosen heuristic of the new dataset (i.e., see Algo. 3). The reason for performing clustering is that it will provide the ranks of algorithms based on the performance of the classifiers for different datasets. When a new problem is encountered, EHHR predicts the performance using HAGA instead of running all classifiers. HAGA reuses the average performances of classifiers on previously solved problems to predict the performance of an unseen dataset. Later, a cluster is assigned to this unseen dataset based on predicted average performance for the recommendation of an ML classifier. The time spent collectively executing HAGA and clustering for an unknown dataset is significantly less than executing and evaluating individual classifiers on that dataset. It chooses a classifier that is less time-consuming for a short-text dataset.
For cluster assignment to the unseen/test data, the euclidean distance is calculated between the centroid of the clusters and the predicted performance of the unseen dataset. The euclidean distance dðu i ; c i Þ (i.e., see Eq. 6) is the square root of the sum of the squares of the differences between the predicted performance (u i ) of unseen dataset and cluster centroid (c i ) in each dimension [54]. The dataset is assigned the cluster for which the distance is minimum.
3.5 Example of EHHR

Materials and methods
This section provides the experimentation details for the proposed EHHR framework.

Experimental setup
This section presents the experimental setup and parameter settings for the framework. Section 4.1.1 explains the dataset used and its experimental settings. Section 4.1.2 provides embedding and their setup.

Benchmark datasets and preprocessing
All the experiments were carried out on ten balanced and imbalanced short-text datasets. Jigsaw comment and Landslide datasets have been collected from UCI [55] and the GRAIT-DM project [56], respectively. All the remaining datasets have been collected from Kaggle [57]. These datasets contain short-text reviews, tweets, or SMS. Table 2 provides the datasets' details, including the size and percentage of positive and negative examples. Senti-ment140 and IMDB datasets contain an equal distribution of positive and negative examples. Landslide, Jigsaw comments, DrugsCom, Amazon baby reviews, Google playstore reviews, and Women clothing datasets depict high skewness in data distribution. The datasets were modified and cleaned as a first step in the framework. Multiclass datasets were mapped to binary classes. The examples with neutral sentiment labels were removed for the multiclass datasets, such as Google playstore data. The reviews and rating columns for the datasets, such as DrugCom, were retrieved. We manually observed the samples to consider them either low for lower satisfaction levels or high for higher satisfaction. We took a random sample of 200 examples for binary classification and analyzed the comments. The rating values in the dataset were in the range of 1-10. By observing the user comments, it was found that 1-4 rating comments were not good, so we considered them low ratings (i.e., 0-label). Since the remarks with a rating of 5-10 seemed favorable, we assigned them a high rating (1-label). The general text cleaning tasks included the removal of punctuation marks, HTML tags, and stopwords. In addition, each dataset was

Word embedding
EHHR was evaluated with two different models (EHHR BERT and EHHR TfIdf ) based on word embeddings (BERT and TF-IDF). For EHHR BERT , the word embedding of each dataset was extracted through the pre-trained model of BERT BASE (Bidirectional Encoder Representations from Transformers) model [58]. BERT BASE architecture contains 12 encoder layers, 768 hidden layers, and 12 attention heads. The BERT provides embeddings of each word with 768 dimensions. The bidirectional nature of BERT allowed the framework to learn the lexicon of words in the shorttext corpus. Hence, BERT provided a deeper understanding of word perspective than sequential models. The machine learning classifiers were added atop the CLS tokenś transformer output.EHHR TfIdf was implemented by extracting word embedding Term Frequency-Inverse Document Frequency (TF-IDF) technique. It is a numerical statistic designed to represent the significance of a word in a collection or corpus of documents [59].

Features extraction
The purpose of the FEM is to find out the features presented in Sect. 3.1 for each dataset. Each dataset was passed to the FEM to extract the features. Lexically sophisticated features of each short-text dataset were retrieved with the help of lexical tokens. For calculating lexical variation descriptor, BNC (British National Corpus) wordlist [60] served as a basis to determine the verb, noun, and adjective types in tweets, comments, and reviews in the datasets. The BNC list contains 29 different families of words retrieved from British and American English to assist vocabulary analysis of the text.
The readability tests were calculated with two variables: average sentence length and average word length. The IMDB dataset showed the highest reading ease score, while DrugCom scored the lowest. For calculating the percentage of hard examples in each dataset, a neural network was set up with 100 hidden layer elements, 0.0001 value for regularization parameter, and a constant learning rate. The top 50 frequently occurring words in each short-text dataset were retrieved to compute the average entropy of frequent items. The sample size for calculating the average TFIDF values was also specified as 50 most frequent words for each dataset.

Heuristics and parameter setting
The heuristic space of EHHR used Support Vector Machine (SVM), K Nearest Neighbour (KNN), Random Forest (RF), Gaussian Naive Bayes (GNB), Decision Tree (DT), and Neural Network (NN) were chosen as Machine Learning (ML) heuristic models. To implement and execute these models, we used Scikit-learn, numpy, and panda python libraries. All datasets were evaluated for the same parameter settings of ML models. Each dataset was split into 70 ratios, 30 for training and testing tasks. The parameters are trained using the grid search strategy. The details of heuristic-wise parameters are given in Table 3. KNN classifier is executed for k range between 3 and 31 (i.e., with a step-size 2). We have used a fully-connected feed-forward NN that takes BERT or TFIDF embedding as an input. NN is searched with tanh, relu, and logistic activations, along with lbfgs, sgd, and adam sovlers for weight optimization. NN shows the best performance for the combination of Rectified Linear Unit (relu) and

Results and discussion
This section presents the results obtained from the experiments and their discussion.

Heuristic space
As explained in Sect. 4.1.4, the heuristic space of EHHR used SVM, KNN, RF, GNB, DT, and NN. Table 4 shows the best parameter values achieved for each model. For KNN, K=3 turned out to be the best parameter setting for the majority of the datasets in the case of EHHR BERT and EHHR TfIdf (i.e., see Table 4). NN shows the best performance for the combination of Rectified Linear Unit (relu) and Stochastic Gradient Descent (sgd) with 0.001 L2 penalty and 0.001 learning rate for both versions of EHHR. RF depicts its best performance for tree depths equal to 16 and 32 for EHHR BERT and EHHR TfIdf , respectively. The best tree depths for DT are 32 and 64 with entropy selection measures for EHHR BERT and EHHR TfIdf respectively. Table 5 provides heuristic's macro f1-score (i.e., first row) and micro f1-score (i.e., second row) achieved for each dataset for the best values of parameters with EHHR BERT . The average performance of all heuristics for each dataset is the highest for the Landslide dataset (i.e., equal to 92% for macro f1-score). The Women clothing dataset achieves a 64% average performance, which shows that it is difficult to predict compared to other datasets. The difference between the heuristic's average micro and average macro f1-scores remains below 4% for Reddit, IMDB, Corona tweets, and Sentiment140 datasets. It is because the training sample fairly represents the positive and negative classes. The difference between the heuristic's average micro and macro f1-score is significant for the remaining datasets. The differences arise as the micro and macro f1scores are calculated differently. The micro f1-score assigns equal weight to each sample, and the macro f1score assigns equal weight to each class. Table 5 also demonstrates the average performance of heuristics in terms of macro and micro f1-scores for all datasets. Neural Network outperforms other used heuristics in terms of macro and micro f1-score. The macro f1-score-based performance of NN is in a range of 4-23% better as compared to other heuristics. The performance of SVM is similar to RF. The micro f1 score for RF is better than DT, SVM, KNN, and GNB within a range of 2-16%. GNB performance in macro f1-score is less than all other heuristics for all datasets except the Sentiment140 dataset. Table 6 provides the performances of heuristics for datasets with the best values of parameters using EHHR TFÀIDF . The average performance of all the heuristics on the Landslide dataset is high. The Sentiment140 dataset is proven to be complex data for most heuristics. The heuristic's performance on the Women Clothing dataset is similar to the Sentiment140 dataset. In the case of TFIDFbased embedding, both datasets are difficult to predict. The difference between the heuristic's average micro and average macro f1-scores remains below 3% for Reddit, IMDB, Corona tweets, Google Playstore, Landslide, and Sentiment140 datasets. For the remaining datasets, this difference is above 6%. The performance of the KNN heuristic is acceptable only on Drugs and Jigsaw datasets. In contrast, the performance of GNB is poor on Drugs and Jigsaw-comments datasets. Macro f1-score of NN is better than other heuristics in a range of 2-18%. On the other hand, the micro-f1-based performance of SVM is highest, i.e., in a range of 1-13%. Tables 5 and 6 show that NN model performs better than other models for both TFIDF and BERT-based embedding. From the point of view of datasets, on average, the performances of the heuristics on the Landslide dataset are better compared to other datasets. In the case of TFIDF, the performance of SVM and RF models is comparable to that of NN. It is because SVM performs effectively in highdimensional space (i.e., embedding is high-dimensional). Moreover, the tree ensembling strategy of RF minimizes the overfitting and provides improved performance [61,62]. The difference in performances of heuristics on different datasets strongly supports the theory of ''No Free Lunch'' [23]. The theorem asserts that no heuristic consistently performs well across all datasets. Furthermore, there are different skewness levels for each dataset. Thus, heuristics become biased when training with the imbalanced dataset [63]. The average performance of the five heuristics across all datasets is superior for TFIDF than for BERT. The only exception is the KNN algorithm which shows better results with BERT embedding. However, only the KNN algorithm exhibits better performance with BERT-based embedding.

Performance of the evolutionary module
The HAGA's (i.e., see Sect. 3.3) application on datasets helped predict the average performance of a classifier. The HAGA chromosome is an 18-dimensional vector, where each gene represents a feature (i.e., from 18 features) of each dataset (see Table 7). All the features are normalized within a range of 0 and 1 and serve as genes of a chromosome. The number of maximum iterations is kept very large to ensure that convergence should be achieved before reaching the maximum iterations. In the worst case (no convergence), the maximum number of iterations will be served as the termination criteria. The loss value for each dataset is calculated by subtracting the actual average macro f1-score from the predicted average macro f1-score (i.e., see Eq. 5). The Final Loss value achieved is 1.9, with a standard deviation of 0.3. Hence, the HAGA can predict the average macro f1-score with an error margin of 1.5 to 2.3. Therefore, without training, we can predict the performance with marginal error. Furthermore, this reduces the time for the choice of the classifier for a problem and helps overcome the brute force searching of heuristics for a new problem.  Reddit  67  77  82  84  73  63  74   68  77  82  84  74  64  75   Women Clothing  59  58  76  73  54  37  60   75  84  83  86  79  37  74   IMDB  61  69  75  79  61  52  66   61  69  75  79  61  62  68   Google Playstore  75  78  75  79  67  52  71   81  80  78  84  76  76  79   Corona tweets  61  67  66  69  62  56  64   62  70  67  70  65  63

Influential features predicted by HAGA
The HAGA in the EHHR framework also predicts the importance of the features of the dataset . The table 8 shows the importance score for the top five features. The importance score validates the dataset descriptors presented in section FEM. The highest importance score achieved by the average entropy proves it is the most important descriptor of data for solving DT and RF algorithms. The variety in verbs and adjectives has also been among the top five descriptors. The number of hard examples can facilitate determining the complexity of various datasets and the effect of data complexity on the classifier's performance.

Convergence of the HAGA
To validate the performance of the HAGA, the experiments were run ten times. Figure 5 shows the finally converged values for loss achieved for each run. The difference in converged loss value in ten runs remained within a range of 0.055 to 0.059 in case of BERT-based embedding, resulting in minute difference of 0.004. For TFIDF, the difference range was 0.045 -0.050. The difference in final loss computed during each run of HAGA in EHHR BERT is less than the final loss to HAGA in EHHR TFIDF . Figures 6a  and 7a show the comparison of convergence of HAGA between a single run and the average of multiple runs. The x-axis shows the number of iterations, and the y-axis depicts the loss values reached in each iteration. The closeness of the two series in both graphs validates the persistent performance of the proposed technique. Furthermore, Figs. 6b and 7b depict the variance on a logarithmic scale among ten different runs of HAGA using BERT-based and TFIDF-based EHHR, respectively. We show that the variance between multiple runs is small, which shows the consistency of the proposed technique.

Correlation among influential features
The heatmap presented in the Fig. 8 shows the correlation among the five most influential features. The smaller correlation value between H p and E avg shows the weak linear relationship between the percentage of hard examples and the average entropy of the frequent words in short-text data. The correlation between the VV-II and AdjV is 0.52, which confirms that both are positively correlated and influential in determining the class of a sample in short-text data. MLWS shows a negative correlation with VV-II and AdjV. The heatmap analysis depicts that 38% of the variation in AdjV is negatively correlated to MLWS, and approximately 8% of the decrease in VV-II is associated with an increase in MLWS. Furthermore, our analysis showed that the two influential features (MLWS and  Reddit  85  85  87  84  53  81  79   85  85  87  84  57  81  80   Women Clothing  65  70  76  76  63  69  70   78  86  88  86  81  76  83   IMDB  67  80  83  82  65  80  76   67  80  83  82  65  80  76   Google Playstore  80  85  85  89  63  75  80   83  88  89  92  65  78  83   Corona tweets  75  77  75  75  60  71  72   77  79  78  76  66  Average Entropy), with the highest importance values, have a strong inverse relationship.

Comparison of HAGA with standard genetic algorithm
In order to validate the performance of HAGA, it is compared with a standard genetic algorithm. The standard GA is implemented with a single-point crossover, fixed mutation rate of 0.02, and tournament selection strategy. Figure 9 illustrates the final loss values achieved for HAGA for EHHR BERT and EHHR TfIdf and standard GA over 500 iterations. The standard GA is evaluated for BERT and TFIDF embeddings results. The curves of standard GA in both cases show notably large differences in final loss values. In both models, HAGA's convergence curves remain below the standard GA's convergence curve, thus resulting in predicting performance close to the actual performances. Importantly, the overlapping of the curves for HAGA with EHHR BERT and EHHR TfIdf demonstrates the proposed framework's consistent performance. Furthermore, due to the adaptive mutation rate, HAGA explores and exploits a diversified population compared to standard GA. Table 9 shows the difference between the performance predicted by a standard GA and the proposed HAGA for both models of EHHR. The predicted performances of HAGA for both models show that HAGA outperforms standard GA in 8 out of 10 datasets (i.e., see Fig. 10).

Data clustering
After applying the fuzzy C-means clustering, two clusters have been identified. In the case of EHHR BERT , the clusters are generated with an average macro f1-score greater than 68% (i.e., for cluster 1) and less than or equal to 68% (i.e., cluster 2). Table 10a and b show the heuristic' ranking for each cluster for EHHR BERT . Table 11a and b display the heuristic' ranking for each cluster for EHHR TfIdf . In the case of EHHR TfIdf , the clusters are obtained with cluster centers at 78.79% and 70.62%. The ranks are computed based on the macro f1-score performance. ''Actual'' is the cluster assigned to the dataset based on actual performance, while ''Predicted'' refers to the cluster assigned to the dataset based on the predicted performance obtained by the proposed model. The clusters obtained from the predicted performance are similar to those obtained from actual performances in both versions of EHHR, except for the IMDB dataset as shown in Table 12. These results verify that EHHR performance remains equally good when different embedding are used.
The research findings indicate that EHHR performs similarly for different word embedding and can effectively anticipate the best-performing short-text classification   heuristic. An important advantage of the EHHR is that it uses a high-level strategy with little or no technical knowledge of low-level heuristics. In contrast to expert opinion-dependent techniques [30,31], EHHR predicts the best heuristic for short-text classification automatically, thus minimizing the need for human involvement. Unlike the techniques proposed in [26,29], EHHR does not heavily rely on an accurate selection of features and parameters for evaluation. As EHHR uses the previously solved problems to guide the search for the best heuristic for short-text, enabling the heuristic prediction to be less time-consuming. However, the heuristic evaluation module needs to execute all heuristics for classifying short-text data only once.

Conclusion and future work
Choosing a classification heuristic is important when applying a classifier to a new dataset or new instances of an existing dataset. Additionally, the heuristics' performance varies per problem. The existing research provides the solution to the abovementioned problem for numeric data. However, a classifier recommendation mechanism that exists for short-text data is lacking. The paper overcomes this limitation for short-text datasets with the help of hyperheuristic. It presents a framework with feature extraction, heuristic evaluation, evolutionary and memory modules to calculate and obtain heuristic performances. Furthermore, the memory module stores and reuses the previously solved short-text classification problems. The proposed Evolutionary Hyper-heuristic-based Recommender framework EHHR is evaluated using six different machine learning  In the future, we want to add and evaluate more shorttext datasets from different domains such as biology and software engineering to our framework. It will aid in broadening the scope of EHHR's applicability to other domains. Fig. 10 Comparison of actual average macro f1-score for each dataset to the predicted average macro f1-score of HAGA and Standard GA

B.2 Lexical sophistication
The percentage of unconventional words in the data refers to the Lexical sophistication. Lexical sophistication 1 and 2 are computed for datasets in EHHR.
1. LS1: LS1 is calculated as the ratio between the measure of W slx (sophisticated lexical words) to W lx (i.e., total lexical words, Eq. B2 ).
2. LS2: The proportion of the count of sophisticated word types(X s ) to count of word types (X) as presented in Eq B3.

B.3 Lexical variation
For computing the extent of vocabulary usage is calculated with the help of lexical variation of words, verbs, nouns and adjective in the dataset.
2. Verb variation-II (VV-2): It is calculated as the ratio between the count of verb types count to the total verbs count in a short-text (Eq. B5).
3. Noun variation (NV): The ratio of nouns usage in writer's comments to the lexical words in corpus(Eq. B6).
4. Adjective Variation (AdjV): It is the ratio of adjectives used in writer's comments to the lexical words in corpus (Eq. B7). It is important to note that the use of adjectives make a short-text positive or negative.
5. Type-token ratio (TTR): Type-token ratio (TTR) is obtained by dividing the number of word types by the number of words present in the text (Eq. B8). Types are the unique lexical elements used in a corpus.

B.4 Lexical diversity
Lexical diversity (LD) measures the vocabulary range adopted in the corpus. Measure of Textual Lexical Diversity (MTLD): MTLD helps to valuate the lexical diversity of the short-text irrelevant to corpus length. It is ''the mean length of sequential word strings in a text that maintain a given TTR value'' [70].

B.5 Lexical richness
It is the assessment of text richness with the help of lexical diversity measures. Another measure of lexical richness you may use is Hapax richness, defined as the number of words that occur only once divided by the number of total words. To calculate this, simply use a logical operation on the document-feature matrix to return a logical value for each term that occurs once and then sum up the rows to get a count. Last but not least, calculate it as a proportion of the overall number of words (ntokens) for better interpretation within your overall corpora. Hapax Richness: It determine the proportion once occurring words(R 1 ) in the corpus.
B.6 Lexical readability The readability of a corpus is quantified keeping in view the length of sentence and words. Two formulae are used in the proposed framework considering mean of number of words in a sentence(i.e. sentence length L s ) and average syllables per word (word length L w ). The L s is computed by dividing the number of words by the number of sentences in a corpus. L w is generated as a result of dividing the number of syllables to the number of words. The coefficients of the reading ease scale involve the systematic selection of word samples from data [71].

B.7 Data quality
Spelling mistakes: Proportion of the misspelled words (M) in the corpus (Eq. B13) is determined to assess the quality of the short-text in the framework.
Spelling mistakes ¼ M X ðB13Þ

B.8 Word length
Mean Length of Word Strings: It (Eq. B14) is the average syllables per word (word length L w ).
Cluster Computing Author contributions All authors contributed to the study conception. Methodology design, data collection, analysis and visualization were performed by BA. The first draft of the manuscript was written by BA and HM and KUK commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Funding The authors declare that no funds or grants were received during this research.
Data availability The datasets analysed during the current study are available from the corresponding author on reasonable request.

Declarations
Conflict of interest The authors declare no conflict of interest.
Consent to participate: Not applicable