{'fraction_non_alphanumeric': 0.02894948717051143, 'fraction_numerical': 0.005740889837924879, 'mean_word_length': 5.056391774432902, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improved Preterm Prediction Based on Optimized Synthetic Sampling of EHG Signal

Preterm labor is the leading cause of neonatal morbidity and mortality and has attracted research efforts from many scientific areas. The inter-relationship between uterine contraction and the underlying electrical activities makes uterine electrohysterogram (EHG) a promising direction for preterm detection and prediction. Due the scarcity of EHG signals, especially those of preterm patients, synthetic algorithms are applied to create artificial samples of preterm type in order to remove prediction bias towards term, at the expense of a reduction of the feature effectiveness in machine-learning based automatic preterm detecting. To address such problem, we quantify the effect of synthetic samples (balance coefficient) on features' effectiveness, and form a general performance metric by utilizing multiple feature scores with relevant weights that describe their contributions to class separation. Combined with the activation/inactivation functions that characterizes the effect of the abundance of training samples in term and preterm prediction precision, we obtain an optimal sample balance coefficient that compromise the effect of synthetic samples in removing bias towards the majority and the side-effect of reducing features' importance. Substantial improvement in prediction precision has been achieved through a set of numerical tests on public available TPEHG database, and it verifies the effectiveness of the proposed method.


I. INTRODUCTION
P RETERM birth, defined as babies born with gestation age less than 37 weeks, is a major and growing challenge for public health systems. Nearly 15 millions babies, or about 10% of total world-wide births, are born prematurely each year. About one million of these preterm babies die because of complications following the preterm birth [1]. Currently, the lack of comprehensive understandings of the uterine contraction initiation mechanism hinders effective early-stage treatment of preterm birth. Once delivery starts, it can not be prevented. Thus, early detection and preemptive treatments are a promising direction for preventing premature babies. Frequently used preterm diagnosis methods include Tocogrametry, Intra-uterine Pressure Catheter, Fetal Fibronectin, Cervical length measurement etc, but none of theses provides reliable results [2].
The expulsion of a fetus is a direct consequence of strong periodic uterine contractions, which results from the generation and propagation of action potentials [3]. The corresponding electric signals can be recorded by electrodes placed on the abdomen of pregnant women, using the electrohysterogram (EHG) technique. Due to the close relation between uterine contraction and the underlying electrical activities, EHG provides a new direction for the development of preterm diagnosis method [4], [5].
Taking advantage of the recent progress in machine learning, a set of new preterm diagnosis methods have been proposed [6]- [8]. Overall, the preterm diagnosis can be categorized as a classification problem, i.e. to decide or to predict a patient (pregnant woman) is at the risk for preterm birth, Manuscript  based on a set of physical examination data (sample) and the features contained therein. It is well conceived that both the abundance of the sample w.r.t. different classes and the quality of the features that distinct different classes are vital to achieve satisfactory classification results.
In recent years, the TPEHG (Term Preterm EHG) database has been widely used for training and testing variant machinelearning-based preterm diagnosis methods. Although there are millions of preterm babies world-wide, the fraction of preterm birth is quite small, compared to the total number of births. This fact is reflected in the composition of the TPEHG database which is a public available database that contains 300 EHG samples of pregnant women [9] . It is noticeable that only 38 EHG samples are collected from patients whose pregnancies will result in preterm delivery, while the other 262 EHG samples are from patients with normal term delivery. Due to the strong difference between the number of preterm and normal delivery samples, applying conventional machine learning algorithms with such extremely imbalanced data will tend to classify the minority samples into the majority class, i.e., there exists a bias towards the majority [10], which is likely to result in inaccurate diagnosis result.
Learning from imbalanced dataset is very active research topics in the field of machine learning [11], [12]. The state-ofthe-art research methods to deal with imbalanced data mining problem can be categorized into two directions: 1) oversampling the minority class or under-sampling the majority one in order to compensate the imbalance of samples between classes to be identified; 2) synthesizing artificial samples from minority class. The former is of limited use when the size of the dataset is small.To be specific, under-sampling could significantly reduce the number of samples to be used in training the learning model, potentially leading to under-estimation; while over-sampling could magnify the feature variations of training samples, possibly resulting in over-estimation.
On the other side, the synthetic sampling with data generation methods aims to generate synthetic data that originated from the minority class. The synthesizing procedure mimics the random distribution of sampling data in the feature space of minority, so that the generated samples are assumed to be close to the actual distribution of minority in its feature space. Including these samples as the minority training set eliminates the imbalance in the original dataset, and removes the classification bias towards the majority. The frequently used synthetic algorithms such as SMOTE [13] and ADASYN [14], have exhibited certain advantages in real applications of preterm diagnosis [7] and other problems [12], [15].
Since both the abundance of training examples and the quality of features are key factors to improve the precision of classification [16], it is also important to extract new features from EHG signals, so that the performance of machinelearning-based preterm prediction algorithms can be improved by combining these new features [17]- [20]. However, notice that when new features are adopted in the training process, the effect of imbalance may deteriorate [21]. Also, when more synthetic/artificial data of the minority class are generated, the representation ability of the features may change. In addition, with the increase of synthetic samples, the noise in the original samples might intensify. Being trained with these dataset, the classifier would overfit [22]. Therefore, adding synthetic samples may affect in a complex way the quality of the features used by the algorithm, and as a result, may alter the classification performances. Although there are works devoted to optimize synthetic algorithms [23], [24], to the best of our knowledge, few work concerns the effect of optimal number of synthetic samples to classifier accuracy. For this reason, it is necessary to explore the relation between the amount of synthetic data and the quality of features and investigate some unified formulation.
This paper investigates the relation of bias elimination and feature importance reduction when introducing synthetic samples in the training process. We determine an optimal minority dataset synthesizing strategy by quantifying feature importances and classification precision of synthetic samples. The rest of the paper is organized as follows: In Section II, we introduce some basic factors regarding the problem, and analyze the underlying principles of the prevalent synthetic algorithms SMOTE and ADASYN, from which we demonstrate the importance of finding an optimal inter-class sample ratio in dealing with learning from imbalanced dataset. Section III further quantifies the effect of synthetic samples and formulates the problem of determination of optimal sample balance coefficient. In Section IV, we verify the effectiveness of the proposed method by applying it to EHG based preterm prediction in a numerical way. Section V concludes the paper.
II. PROBLEM STATEMENT As explained in the introduction, the strong imbalance between pathological and normal outcomes from the available database results in possible inaccuracies in the classification algorithms. To avoid this, the preferable machine-learningbased algorithms typically introduce a certain amount of synthetic preterm sample data to mitigate the bias towards majority (normal delivery). However, the possibility of misclassifying term samples increase at the same time. Thus, how many synthetic samples should be added is directly related to the performance of a machine-learning based diagnosis method.
A. Sample balance coefficient and feature score As stated in Section I, the abundance of training samples of all classes is essential to the performance of classification methods. For instance, the aforementioned TPEHG database contains great more normal term samples than the preterm ones (262:38), so it is natural to utilize data synthesis techniques to generate samples of the minority class, i.e., the artificial preterm sample. However, there is a lack of understanding about how many minority class samples should be synthesized without changing the number of majority class samples, and what the after-effect will be w.r.t. the classification performance if we synthesize more than enough samples of minority class.
The enrichment of minority class samples by applying data synthesis techniques will improve the classification performance in certain sense, however, it is worth noting that introducing synthetic data might alter the original pattern of sample distribution in the feature space, depending on the underlying mechanics of synthesis, i.e., the boundary in feature space between different classes might be blurred. To better quantify the contribution of different features to classification, we introduce the following feature score f i s defined as in [25], (1) where x + k,i and x − k,i denote the measured physical value of feature i from sample k that is in positive (minority or preterm ) class and negative (majority or term ) class, respectively.x i is the average value among all samples,x + i is the average value of all n + positive (minority or preterm) samples, and x − i is the average value of n − negative samples. According to (1), f i s records the discrimination between minority class (+) and majority class (-) by counting how divergence of the samples are in the feature space. It is worth mentioning that the feature score explicitly relates to the size of testing samples. We introduce the sample balance coefficient α by, where n + and n − are the numbers of samples in the minority and majority class after applying synthetic algorithms, respectively. Notice that n + include the number of synthesized data. Also, for any specified classification problem, different features abstracted from samples jointly contribute to the final classification result. According to (2) and (1), it is reasonable to define the global feature score F score as the weighted sum of different feature scores f i s , i.e., where the weights w i ∈ [0, 1] are introduced to represent the importance of feature i to the classification, and N is the number of features used in the final classification. By construction, the definition of (3) links the the number of synthetic samples and the quality of the features. It provides a unified performance metric which is essential for further investigation.
To proceed, all features are initially used to build a forest, from which we obtain a value of classification accuracy [26]. The reduced classification accuracy by randomly permuting a node in the tree gives a reliable measure of the feature's importance [27].

B. Size of the synthesized data and distinguishability of the features
After introducing the definitions of sample balance coefficient α and feature score f i s / F score (α), it is convenient to investigate the attributes of the conventional data synthesis algorithms such as SMOTE or ADASYN, and consider their feasibilities in the application of preterm diagnosis using the TPEHG database.
Although data synthesis algorithms tend to mimic the natural distribution of sample in its feature space, the nature of random synthesis of minority samples inevitably has certain effects on features' ability to discriminate different classes. For instance, ADASYN tries to synthesize more data from minority samples surrounded by more majority [14]. As shown in Fig.1 (a), the synthetic samples are more likely to appear on the left due to more majority data samples around each minority ones. Although they intend to ease classification by emphasizing on samples that are hard to learn from, at the same time, they would potentially make originally separable datasets non-discriminated in the feature space.
Contrary to ADASYN, SMOTE does not take into account the surrounding of the minority samples. For any minority sample s i , it randomly selects another minority s k among its k th nearest neighbors and synthesizes an artificial samples As a result, the synthesized samples will concentrate in the region containing more minority samples (see Fig.1(b)), which implies that it maintains the original distribution pattern, without diminishing features' contribution to classification.
To see the effect of synthetic sampling on features' contribution to classification, we apply the aforementioned two synthetic algorithms to TPEHG database with frequently used features: 1) the root mean square value of the signal (rms); 2) the median (F med ) and peak (F peak ) frequency of the power spectrum; 3) the sample entropy of the signal (E samp ) extracted from EHG signals. We use hose with the gestation age less than 37 weeks as the preterm (minority class) samples. Fig. 2 shows the variation of features' contribution to classification represented by its f i s after synthetic sampling. It is evident that the peak frequency gives the highest feature score f i s among these four features. Its effectiveness for classification has been confirmed by other authors [2], [28], [29]. Also, it is quite astonished to see that both techniques tend to deteriorate features' ability in separating samples. SMOTE shows more superiority, i.e., the feature scores are higher after applying  SMOTE than those after applying ADASYN. This is consistent with the previous analysis of data synthesis mechanics of SMOTE and ADASYN. It is also worth noting that features' classification ability is sensitive to the number of synthetic samples introduced. This can be seen from Fig. 2, which shows the featured scores f i s after adding synthetic data with different sample balance coefficients α (see Eq. (2)) α = 0.3, panel (a) and α = 0.5, panel (b). As shown by the figure, adding synthetic samples degrades the capability of the algorithm to distinguish the various features. However, synthetic samples are required to eliminate classification bias against the minority. As a result, a trade-off should be found between the number of synthetic samples and features' quality, in order to optimize the final performance of classifiers trained with these data.

COEFFICIENT
As discussed in Section II, to accomplish a machinelearning-based preterm diagnosis, synthetic sampling with data generation of the minority class is inevitable, and the balance between synthesized data and the feature quality must be considered. Intuitively, increasing the number of minority samples by generating synthesized data should increase the prediction precision on the minority class and reduce the bias towards the majority. On the other hand, the prediction precision on majority class may fall if there exists too many minority samples. Ideally, we would expect a no-bias learning system when the sample balance coefficient α = 1.0. In real applications, however, due to the imbalance of the available original samples between classes, the optimum may differ from α = 1. To this end, we introduce two functions C + and C − describing the putative biases induced by the sampling on the minority and the majority class, respectively, In the present work, we chose the parameter k = 4 to describe how the prediction is affected by the balance coefficient α. In Eq. (4), α 0 is the original sample balance coefficient before generating any synthetic data.
Eq (4) implies that when α is smaller than 1, the majority is well described, and C − is close to 1. On the other hand, when α is large, the minority is accurately described (C + is close to 1), but the majority will be affected (C − is reduced).
For most of the classification purposes, a high precision on different classes is demanded (no bias towards any class). To this end, in the case of learning problems from imbalanced data samples, it is required that minority training examples are re-sampled to match the number of majority, i.e., α = 1, in order to remove potential bias towards majority. However, recall the synthesis procedure, additional training examples are generated from the original minority data sets and can be seen as a kind of "copy" of the original data. As a consequence, adding too many synthetic data improves only the precision on the minority class, while degrading the accuracy on the majority classification.
To account for these effects, we introduce the constant α 0 in the definitions of C + and C − in Eq.(4). This equation expresses that when there are enough original minority samples (α 0 → 1), constant α 0 goes to 0. Thus there is no need to synthesize training examples. When learning tasks from imbalanced dataset are encountered (α 0 < 1), the optimal balance coefficient α deviates from the ideal value.
To take into account the bias described above, we simply multiply the effective feature score, F score , as defined in Eq. (3), by C + × C − , to come up with an effective score, F e score . Combining the requirement on high prediction performance on preterm as well as term, we determine the optimal sample balance coefficient α * as follows,

IV. EXPERIMENT VERIFICATION
The above analysis suggests a way to improve preterm diagnosis precision by determining the optimal sample balance coefficient without sacrificing the precision on term prediction. In this section, we provide experimental results to verify the effectiveness of the proposed method, particularly, we propose the numerical way to the determination of α * . The general procedure regarding the experiment is summarized in Fig. 3 Fig. 3. Flowchart for determining effective feature scores. After having synthesized artificial samples, calculate the weight w i and score value f i s of each feature. Based on these two values, the weighted feature score Fscore is defined. Combining the activation/inactivation functions with Fscore gives effective feature score F e score .

A. Material and Terminologies
Electrohysterogram (EHG) data obtained from the Physionet database (TPEHG) are used. Root mean square (rms), peak (F peak ) and median (F med ) frequencies and sample entropy (E samp ) are extracted from the recorded EHG signals. Based on the recorded gestation age (T gest ), 300 samples are spilt into two groups with the criteria T gest < 37 weeks, which gives a minority class ( preterm ) of 38 samples and a majority group (term) of 262 samples.
Recalling the purpose of predicting gestation status, frequently used simple but powerful classifiers, like Support Vector Classifier (SVC), Linear Discriminant Classifier (LDC), Logistic Regression Classifier (LRC), Decision Tree Classifier (DTC), Gradient Boosting Classifier (GBC) etc., are used to verify the proposed method. Following [29], Holdout Cross-Validation with 80% of the whole dataset is designated for training the classifiers and the rest 20% for testing, from which we calculate sensitivity (True Positive (preterm) Rate, TPR) and specificity (True Negative (term) Rate, TNR). Notice that these two quantities alone can not well represent our requirement on high performance of both positive and negative prediction. For this purpose, we introduce G mean and Overall Accuracy (OA) as the performance metrics as follows, where TP and TN are correctly predicted positive and negative test samples, FP and FN are incorrect predictions, respectively. Apparently, classifiers provide the most accurate prediction on both preterm and term classes are typically with the highest value of G mean and overall accuracy OA. Beside these quantities, Area Under (Receiver Operator) Curve (AUC) [30] is also used to verify the proposed method.

B. Optimal synthetic preterm samples
Based on the intuitive analysis given in the previous sections, SMOTE has stronger capability than ADASYN in keeping features importance in classification. We first apply it to generate synthesized samples of minority class to match a given sample balance coefficient α. Fig. 4(a) shows the measured feature score f i s with different α. Obviously, it is hard to determine how many synthetic samples should be generated, as features have their measured f i s values well separated and vary with α. After having synthesized enough artificial samples, we use the features to build a forest, from which we obtain a value of classification accuracy. The reduced classification accuracy by randomly permuting a node in the tree gives a reliable measure of the feature's importance (weight) [27]. Calculating features' importance (weight) at each α allows us to examine the combined effect of synthetic samples on features' importance F score . As shown in Fig. 4 (b), F score decreases with the increase of synthetic samples, illustrating the drawbacks of the synthetic sampling. This also implies the importance of the determination of the optimal sample balance coefficient α.
Combining the activation and inactivation functions previously introduced ( Fig. 5(a) ), the effective feature score F e score shows a trend that helps us to easily determine out the optimal sample balance coefficient α * . As shown in Fig. 5(b), F e score firstly increases with the increase of α, manifesting the effect of synthetic samples for eliminating bias towards the majority (term) samples. Due to the weakening of features' scores and the bias towards term at large α, when F e score reaches its peak at α ≈ 0.7, it starts to decrease. The position where the peak F e score locates provides the optimal α * . Fig. 5 demonstrates that with the optimal α * , the capability of the various features to distinguish between different classes has not been lost, while reducing the bias towards the majority.

C. Validation
To verify the obtained optimal sample balance coefficient α * , same features extracted from 80% of samples in TPEHG database are used to train a SVC classifier, the other 20% are then used for verification. As training and testing samples are randomly selected, the calculated quantities representing classifiers' performance vary from time to time. For this reason, we repeat the training-testing process for 100 times at each α. Fig. 6 shows the variation of the calculated system performance with the increase number of synthetic samples added to the dataset, i.e., the increase of α. As expected, prediction precision on minority increases, while that of majority decreases. It is worth noting that the two curves intersect at the point α ≈ 0.7, which is the optimal sample balance coefficient α * determined previously. At this point the trained classifier eliminates most of the bias toward the majority (term) and increases the precision on minority (preterm) prediction without sacrificing too much on the precision of term prediction. This is confirmed by the accompanied variation of G mean and AUC, see Fig. 6(b), where both of these two quantities reach their apex at α ≈ α * . The receiver operator characteristic curve (ROC) and the associated AUC values shown in Fig. 7 indicate the cut-off values for the true positive and false positive rates at different sample balance level (different α). Apparently, in the case of optimal sample balance coefficient α 0 , the SVC classifier shows better performance. Comparing to the case of ideal balance (α = 1.0), training with optimal amount of synthetic samples leads to a big improvement in terms of AUC values. The advantages of determining the optimal sample balance coefficient shown in SVC hold for other different classifiers. Table I gives a comparison of frequently used parameters for evaluating classifiers' performance. It can be observed that with the previously determined optimal sample balance coefficient α * , all the classifiers show a great improvement in performance, especially for SVC based classifiers.
The effect of optimal sample balance coefficient also works with ADASYN. Although this method has less power in keeping features' ability in classification, combining the proposed the activation and inactivation functions do give us an easyto-identify optimal sample balance coefficient α * = 0.8. As shown in Fig. 8, the term and preterm prediction accuracy of a SVC classifier trained from these dataset gives the optimal performance at this α * . However, as effective feature score F e score obtained from ADASYN is less than that from SMOTE as indicated in Fig.5, it can be expected that the corresponding classifier after training through ADASYN method is underperformed than that through SMOTE method.

V. CONCLUSION & DISCUSSION
Machine-learning based automatic disease diagnosing system provides a prospective direction for modern healthcare. In these applications, the availability of healthcare data and the effectiveness of features extracted from these samples play a crucial role. However, healthcare data are typically imbalanced, with most of samples being healthy (majority), and a few being disease (minority). Training with imbalanced dataset, classifiers typically introduce biases towards the majority, making the automatic diagnosing system less useful. To solve the problem, data synthesizing algorithms are used to generate artificial samples from the minority. However, this is typically accompanied by reduced ability of features in class separation. In this paper, we propose a method for determining the optimal number of artificial samples should be synthesized. To proceed, we measure features' contributions and their weights in class separation in the case of introducing different amount of synthetic samples. Combining with the activation and inactivation functions introduced to describe the effect of sample abundance on classification precision, we obtain the optimal sample balance coefficient that compromises the effect of synthetic samples on eliminating bias and the side-effect of weakening feature importance. We apply the proposed method to predict preterm behavior using features extracted from public available database TPEHG. After applying synthetic algorithms, system performances are compared under different scenarios and the results highlight the importance of optimal sample balance coefficient proposed in the work.
One might argue that it is more critical that an automatic diagnosis system mis-identifies a real preterm patient than that mis-identifies a term patient, considering the consequences of serious complexities the preterm babies would have. As such, increasing synthetic samples should be of greater interest, which is the case shown in Fig.6 and 8. However, special attention should be paid before drawing this conclusion. Since there is no practical test for any EHG based preterm diagnosis system, its performance is typically verified using data samples randomly chosen from the total sample set. The dataset used to check the performance on preterm prediction are synthesized from the same minority class as those used for the training purpose. With the increase number of synthetic samples included, the validation samples are getting closer and closer to the training samples, which lead to unreal high values of preterm prediction accuracy, especially in real applications [31]. In the proposed method, by introducing the activation/inactivation functions that account for the original size of minority samples, we suppress this side-effect. It is believed that validation results should be close to really applications.