{'fraction_non_alphanumeric': 0.04378158400787085, 'fraction_numerical': 0.019721837127141005, 'mean_word_length': 4.366450683945285, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Imbalanced learning for RR Lyrae stars

We apply machine learning and Convex-Hull algorithms to separate RR Lyrae stars from other stars, like main sequence stars, white dwarf stars, carbon stars, CVs and carbon-lines stars, based on the Sloan Digital Sky Survey (SDSS) and Galaxy Evolution Explorer (GALEX). In the low-dimensional space, the Convex-Hull algorithm is applied to select RR Lyrae stars. Given different input patterns of (u-g, g-r), (g-r, r-i), (r-i, i-z), (u-g, g-r, r-i), (g-r, r-i, i-z), (u-g, g-r, i-z) and (u-g, r-i, i-z), different convex hulls can be built for RR Lyrae stars. Comparing the performance of different input patterns, u-g, g-r, i-z is the best input pattern. For this input pattern, the efficiency (the fraction of true RR Lyrae stars in the predicted RR Lyrae sample) is 4.2% with a completeness (the fraction of recovered RR Lyrae stars in the whole RR Lyrae sample) of 100%, increases to 9.9% with 97% completeness and to 16.1% with 53% completeness by removing some outliers. In the high-dimensional space, machine learning algorithms are used with input patterns (u-g, g-r, r-i, i-z), (u-g, g-r, r-i, i-z, r), (NUV-u, u-g, g-r, r-i, i-z) and (NUV-u, u-g, g-r, r-i, i-z, r). RR Lyrae stars, which belong to the class of interest in our paper, are rare compared to other stars. For the highly imbalanced data, cost-sensitive Support Vector Machine (SVM), cost-sensitive Random Forest and Fast Boxes are used. The results show that information from GALEX is helpful for identifying RR Lyrae stars and Fast Boxes are the best performers on the skewed data in our case.


Introduction
With the rapid development of astronomical observation technologies, various large survey projects of different wavelengths are finished, on-going and in plan, such as 2MASS (Skrutskie et al. 2006), SDSS (York et al. 2000), GALEX (Bianchi et al. 2011), WISE (Wright et al. 2010), LSST (Ivezić et al. 2009) and others. The observed data from different bands have grown dramatically, which promotes the formation of multi-band astronomy.
The observation and research on the multiwavelength nature of celestial objects have been opened up. These multiwavelength data are of great value for the study of objects. Faced with so massive amount of astronomical data, we should extract the useful information from the data by automatic methods.
As a tracer, RR Lyrae stars can be used to study the structure and evolution of the Galactic halo (Bullock et al. 2001). Their absolute magnitude is nearly a constant value, which is better for calculating the distances and refining models to measure the Galaxy's mass as the standard candles (Krisciunas 2001). So it is important to pick out RR Lyrae stars from large survey data. Many works have been performed by the selection criteria in the two-dimensional color space. Davenport et al. (2014) used the Ks − W 3 and g − i color space and selected 199 infrared excess candidates. Finlator et al. (2000) applied the r − Ks and J − Ks color-color diagrams to trace the stellar color locus and classified the main sequence stars. But in a two-dimensional color space, RR Lyrae stars are not easy to be separated better from other sources, e.g. Cepheid variables, quasars, normal spectral class A, F and M stars. Nevertheless Fan (1999) presented the stellar distribution in the simulated color space of (u − g, g − r, r − i) and got an intuitive understanding of the stellar distribution. Krisciunas et al. (1998) focused on the identification of the RR Lyrae stars based on the SDSS colors. Ivezić et al. (2000) selected 148 RR Lyrae star candidates from SDSS commissioning data for about 100 deg 2 of sky. Ivezić et al. (2005) also picked out RR -4 -Lyrae stars with SDSS DR1 and found that part of the candidates were the remnant of the Sagittarius dwarf galaxy disrupted by the tidal force of the Milky Way. Apparently, it is possible to recognize RR Lyrae stars only from their single-epoch data.
Besides the simple color selection criteria, machine learning algorithms are applied to select RR Lyrae stars. Random Forest is of importance to detect members of rare classes.
Many studies for selecting RR Lyrae stars are based on the Random Forest algorithm (Abbas et al. 2014;Hernitschek et al. 2016;Armstrong et al. 2016). Sesar et al. (2017) explored XGBoost to identify RR Lyrae stars from the PS1 sample with the fitting light curves. To construct an automated approach for targeting RR Lyrae stars, Elorrieta et al. (2016) constructed a good performer based on the AdaBoost family of classifiers.
In fact, the rare or unknown cases are our interest in astronomy, e.g. identifying these rare objects in large survey databases, separating RR Lyrae stars from quasars, detecting gravitational lenses from image databases. RR Lyrae stars belong to this class and are the minority compared to the whole star sample. As a result, there is a thorny problem, which is the high imbalance of data. In general, classical machine learning algorithms do not work well with imbalanced datasets, mainly because they attempt to reduce the overall misclassification error which biases the classifier towards the majority class in imbalanced datasets. In order to select the minority, main approaches are raised to deal with class imbalance, including re-sampling or re-weighting, changing search strategies in learning, cost-sensitive learning, one-class learning, using another measures, adjusting classification strategies, using hybrid and combined approaches (boosting like re-weighing) and others.
Sampling methods include oversampling, undersampling and synthesizing new minority classes, like SMOTE (Chawla et al. 2002). The cost-sensitive learning methods are hot in machine learning, and aim to get a minimal cost result on classifying the skewed data (Weiss et al. 2007). Cost-sensitive learning makes the original machine learning algorithm -5adapt to the needs of the real application to some extent, by the cost imbalance of different miscarriage cases in the real world. Support Vector Machine (SVM) with cost-sensitivity, Random Forest with cost-sensitivity and Fast Boxes were applied to solve the imbalance of classes (Rudin et al. 2014). Their results showed that Fast Boxes were better for separating the interesting class from other classes accounting for the majority.
In this work, our primary goal is to separate the RR Lyrae stars from the other types of stars. This paper is organized as follows. Section 2 provides a brief description of the SDSS and GALEX surveys. How to get the samples is also presented in detail. In Section 3, we describe the algorithms and apply them on the samples. Section 4 discusses the performance of the approaches in our case and analyses the results. Finally, we give the conclusions and the future work. (York et al. 2000) is a major multi-filter imaging and spectroscopic redshift survey using a dedicated 2.5-m optical telescope, which provides a wide field of view, and has five filters located in the u, g, r, i, and z bands, which span wavelengths from 0.36 to 0.90 µm. The processed data include all photometric and spectroscopic observations. Firstly, a sample of stars with spectral classification is produced. To create a reliable -6and complete sample of stars, we need to limit the original SDSS data. Using the DR13 SkyServer and access to the Catalog Archive Server (CAS) database, we extract the point sources with their spectral classification. They are collected from the SpecPhotoAll table of SDSS DR13. Some sources should be removed because either observed multiple times or have spectroscopic data but lack photometry. In our work, we set the class = 'Star' to select the stellar objects. To eliminate the sources observed under bad conditions, we set the SDSS flags with BRIGHT=0, SATURATED=0, EDGE=0, NOPROFILE=0 and BLENDED=0, as provided by Davenport et al. (2014). In addition, the PSF magnitudes used are de-reddened and corrected for Galactic extinction by the dust maps of Schlegel et al. (1998).

SDSS
Imposing sciencePrimary =1 and Mode = 1, we select the sources with best spectra and best photometries. Due to the impact of magnitude errors, there is a large deviation in the distribution of stars in the color-color diagram. In the u, g, r, i, and z bands, the resolution of u band and z band is lower than g, r and i bands, so we set the psfMagErr u <0.2, p.psfMagErr g <0.1, p.psfMagErr r <0.1 and p.psfMagErr i <0.1, p.psfMagErr z <0.2.
Finally 511,201 sources are obtained, which consist of main sequence stars, white dwarf stars, carbon stars, CVs and carbon-lines stars.
The sample of RR Lyrae stars are got from the SDSS Stripe 82. There are 483 RR Lyrae stars (Sesar et al. 2007). The region on the celestial equator, dubbed "Stripe 82", is imaged in more than 10 times and gives co-added optical data 2 times deeper than single epoch SDSS observations. It is superior in studying many time-varying objects, such as variable stars and quasars. The repeated observations also make the photometric data of non-varying stars more reliable.
To identify wether RR Lyrae stars can be separated from other types of stars towards the blue end of light, cross-match of 511,684 sources (483 RR Lyrae stars and 511,201 others) with the GALEX database is performed within a radius of 3.0 arcsec. Finally -7 -169,719 sources are selected and machine learning algorithms are applied to them.

Convex hull algorithm
Convex hull is a concept in computational geometry. In a real vector space V , the intersection S of all convex sets X for a given convex set X is called the convex hull of X. The convex hull of X can be constructed by a linear combination of all points ( X 1 , X 2 , · · · , X n ) in X, which can be imagined as a rubber band just wrapped in all points in the two-dimensional Euclidean space. For more information of convex hull, please refer to Sklansky (1982). Given a set of points on a two-dimensional plane, the convex hull is a convex polygon formed by concatenating the outermost points, which contain all the points in the data set. Convex hull can be constructed for the sample using Qhull 1 , which computes the convex hull for the 2-d, 3-d, 4-d and higher dimensional data. To get a more visual result, we only compute the convex hull in the 2d and 3d spaces.

SVM
Support vector machine (SVM), proposed by Vapnik (1995), is a supervised learning algorithm for classification and regression analysis, which separates the points using a hyperplane. If the hyperplane has the largest distance to the nearest training points of any class, a good separation can be obtained. The superiority of SVM is obvious for 2-class classification. Suppose that classifying n points in real space R m , denoted by the matrix X m×n . SVM predicts the outputs y i = ±1 of input vector x i = (x 1i , x 2i , · · · , x mi , 1) T , i = 1, 2, · · · , n. Finding the optimal hyperplane with a maximum margin by minimizing the following decision function, the training data may be separated by the hyperplane.
where l(y i , ω T , x i ) = max(0, 1 − y i ω T , x i ) represents the loss function, ω is called the normal vector of the hyperplane and C denotes the penalty parameter.
There are many applications of SVM in astronomy. For example, Peng et al. (2013) made a combined classifier of SVM and KNN to select the quasar candidates. Gao et al.
(2009) compared the performance of SVM with K-Dimensional Tree (KD-Tree), and got a better result to separate quasars from stars. In this work, input vectors are mapped into a high-dimensional space to separate RR Lyrae stars from other types of stars.

Random Forest
Random Forest grows many decision trees to classify the data (Breiman et al. 2001).
Given a new input vector, each tree votes for the class. Over all trees, the forest makes a choice and selects the classification having the most votes. The advantages of Random Forest are as follows (Gao et al. 2009): 1. Among many machine learning algorithms, the accuracy of Random Forest is high.
2. It can handle a very large number of input variables efficiently.
3. It estimates which variables are important in the classification problems.
4. In the forest building progress, it produces an inherent unbiased estimate of the generalization error.
-9 -5. It has an efficient way of estimating missing data and remains accuracy when a large fraction of the data are missing.
6. It can balance errors in unbalanced data sets.
7. It calculates the approximations between cases that can be useful for clustering, outlier detection and giving interesting data views.

SVM and Random Forest with cost-sensitivity
Although cost-sensitivity are adopted from scikit-learn (Pedregosa et al. 2011).

Fast Boxes
The recent Fast Boxes algorithm is also used to compare the performance with the baseline algorithms. Fast Boxes are also a cost-sensitive learning algorithm, where relative costs are implicitly determined during training. Data are assumed to lie in a grid. The minority class is clustered and a minimal enclosing box in the grid is determined around each cluster. The feature space is partitioned based on the boxes and finally the boundaries are expanded using an exponential loss function (Shrivastava et al. 2015). More details about the algorithm of Fast Boxes and three stages of Fast Boxes are as follows (Rudin et al.

2014):
1. Characterizing: the positive class is clustered into K clusters, which is a one-class learning method and sets decision boundaries around the clusters.
2. Dividing space: the space is divided into subspaces according to the boundaries of the negative and the positive class.
3. Boundary expansion: using a regularized exponential loss, the boundary is extended to contain more positive class.

Evaluation Metric
In a low-dimensional space, completeness (Recall) and efficiency ( Output: boundaries of boxes 1: Scale the features (different input patterns) to be between -1 and 1.
2: Cluster the RR Lyrae stars into K clusters.
3: Compute the starting boundaries l s,j,k and u s,j,k for each cluster, the j-th dimension lower and upper boundaries for the k-th cluster.
4: Construct local classifiers X l,j,k and X u,j,k .

5:
For avoiding numerical problems, multiply each term by exp(1) and revise the lower and upper boundaries.
6: Perform expansion to guarantee the box is always expanding, which implies that this algorithm includes as many minority classes of samples as possible.
7: Un-normalize by rescaling the input patterns back to get meaningful values. In a high-dimensional space, the different cost ratios (the cost for the majorities) are used to evaluate the performance of machine learning algorithms. For every algorithm, each weight corresponds to a point on the receiver operating characteristic (ROC) curve, which is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. ROC analysis provides tools to select possibly optimal models and to discard suboptimal ones independently from (and prior to specifying) the cost context or the class distribution. Actually, the cost information used in the paper is -13 -

Performance of the algorithms
Our aim is to pick out RR Lyrae stars from the whole star sample. The performance to select RR Lyrae stars is influenced by some factors, such as sample imbalance, the performance of a classifier. The number of known RR Lyrae stars is 483 in our case.
Therefore selecting RR Lyrae stars from other stars belongs to heavy imbalance problem.
Here we plan to explore convex hull to sort RR Lyrae stars in the low-dimensional spaces, and apply cost-sensitive SVM, cost-sensitive Random Forest as well as Fast Boxes to choose RR Lyrae stars in the high-dimensional spaces.
What is more, we compare the performance of convex hull with that of the color cut method in Ivezić et al. (2005). For convex hull algorithm, the efficiency (1.0%) is the best with the input pattern of (u − g, g − r) in a 2-d space, while the efficiency (4.2%) is the best with the input pattern (u − g, g − r, i − z) in a 3-d space. The efficiency of color cuts is 1.7% when we adopt the same color cut method in Ivezić et al. (2005). As indicated in Table   1, the performance of Ivezić et al. (2005) is higher than the convex hull in a 2-d space, while in a 3-d space, the performances with convex hull algorithm are all better than it. As convex hull algorithm with the best input pattern (u − g, g − r, i − z), the efficiency can be increased to 9.9% with a completeness of 97% and to 16.1% with 53% completeness by removing some outliers. In terms of 53% completeness, for two-dimensional input patterns, the convex hulls for RR Lyrae stars are shown in Figures 1-3 Table 2, if including all sparse RR Lyrae stars, the efficiency reduces to the minimum 4.2% while the completeness reaches the highest 100%; the efficiency amounts to 100% but the completeness is equal to 1%. When applying the criteria with higher completeness to select RR Lyrae star candidates, more other kinds of stars will be contained. In general, astronomers are inclined to choose high efficiency at the expense of completeness. Therefore we should compromise efficiency and completeness with caution in the real applications.
In a higher dimensional space, cost-sensitive learning is used. Note that undersampling is a wrapper-based approach that makes any learning algorithm cost-sensitive, while the cost-sensitivity is embedded in the cost-sensitive learning algorithm (Weiss et al. 2007). For our case, the number of the majority is large enough while the minority is too small. It is necessary to reduce the size of the majority class and keep different classes balanced. Therefore undersampling is combined with the cost-sensitive learning to train our sample. Undersampling downsamples the majority class randomly to reduce the ratio of the majority to the minority. We use the cross-matched data of SDSS and GALEX photometric data to single out RR Lyrae stars. The number of cross-matched sources is 169,719. They are randomly split into two parts, whose number ratio is 7:3. From the first part, the training set is generated according to the ratios of RR Lyrae stars to the others, which are 1:10, 1:50, 1:100, 1:150, 1:200, 1:250 and 1:300, respectively. The second part is set as the verification set. The detailed workflow is indicated in Figure 9. For different algorithms, the imbalance weighting parameters of cost-sensitive learning need be adjusted -15to the optimal in order to compare the performance. The weight for the negative class is c, whose value goes from 0.1 to 1 and its step is 0.1. For the cost-sensitive SVM, the 10-fold cross-validation is used to select the optimal γ and C parameters. The γ parameter ranges from 2 −5 to 1, and the C parameter varies from 2 −5 to 2 5 . The cost-sensitive Random Forest also uses the 10-fold cross-validation, which sets the n estimators from 10 to 100.
Like Rudin et al. (2014), the training set for Fast Boxes is separated into 3 folds to select the K and expansion parameters, whose values are 0, 5, 10, · · · , 200. For Fast Boxes, we apply 3-fold cross-validation instead of 10-fold considering the complex computing of Fast Boxes.
To compare the performance of Fast Boxes with SVM and Random Forest better, the and (NUV − u, u − g, g − r, r − i, i − z, r) are adopted. For different patterns, we get different results for every algorithm. Tables 3-6 show the performances in terms of mean

AUH values and their standard deviations. For each input pattern of SVM and Random
Forest with cost-sensitivity, when the ratio of RR Lyrae stars to other types becomes larger, their AUHs become smaller and the performances become worse. But Fast Boxes have a stable performance. All the results show that Fast Boxes are a good performer for highly skewed data. Comparing the AUH of Fast Boxes with different input patterns, that of (u − g, g − r, r − i, i − z) is better than that of (u − g, g − r, r − i, i − z, r) when only considering optical information; (NUV − u, u − g, g − r, r − i, i − z, r) is the best input pattern as given optical and ultraviolet information. They also indicate that added information from GALEX is helpful for identifying RR Lyrae stars. Figure 10 shows the ROC Curve of Fast Boxes with the input pattern (NUV − u, u − g, g − r, r − i, i − z, r).
In a ROC curve, the true positive rate (Sensitivity) is plotted as a function of the false positive rate (100-Specificity) for different cut-off points. Each red circle on the ROC curve represents a sensitivity/specificity pair corresponding to a particular decision threshold. A -16test with perfect discrimination (no overlap in the two distributions) has a ROC curve that passes through the upper left corner (100% sensitivity, 100% specificity). Therefore the closer the ROC curve is to the upper left corner, the higher the overall accuracy of the test is (Zweig & Campbell 1993).

Conclusion
Learning from imbalanced data is still a hot and challenging issue in machine learning, and always faced in reality, such as image recognition, pointed source classification, distinguishing supernovas from large surveys, detecting rare objects or phenomena in astronomy. Since a classifier is aimed to obtain the highest classification accuracy, the classifier is constructed with highly-imbalanced class distribution data set and predicts an unknown sample as the majority class much more frequently than the minority class. deeper magnitudes of celestial objects, which can be thought of as good data testbeds for various algorithms. As the amount of data increases, the training sample is larger and more complete. Therefore the performance of a classifier further improves and the classifier may be used for selecting much more RR Lyrae stars candidates. In addition, other efficient algorithms dealing with highly-skewed data will be explored in the astronomical applications. Moreover we will select the other special objects from large samples with Fast Boxes.
York, D. G., et al. 2000, AJ, 120, 1579Zweig, M. H., Campbell, G. 1993 This manuscript was prepared with the AAS L A T E X macros v5.2.   r, r − i, i − z, r). Each red circle on the ROC curve represents a sensitivity/specificity pair corresponding to a particular decision threshold.