{'fraction_non_alphanumeric': 0.02593999037278708, 'fraction_numerical': 0.009787666470556775, 'mean_word_length': 5.051132686084142, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An ensemble classification based approach for breast cancer prediction

Breast cancer is the second major reason for deaths in women. Early detection of the breast cancer and receiving the appropriate treatment can reduce the death rates as survival becomes hard in the higher stages of the tumor growth. Application of machine learning in healthcare play a key role in aiding the clinical experts to detect the disease at a early stage and perform precise assessment. This paper proposes a pattern recognition methodology that uses breast cancer biomarkers as the attributes and ensemble classification approach for accurately detecting the presence of cancer. The proposed method was evaluated for the samples of the breast cancer Coimbra dataset by fusing the decisions of naive Baye’s, radial basis function neural network and linear discriminant analysis classifiers based on majority voting rule. The experimental results demonstrated the enhanced performance of the system with fusion of classifiers as compared to the single classifiers.


Introduction
An uncontrolled way of cell division leads to cancer and these cells often invade into other healthy tissues. Breast cancer is the most common and is the leading cause of death in women. It can be invasive or non invasive: invasive cancer spreads from the ducts or glands to other parts of the breast while noninvasive cancer does not spread from the original tissue. Based on this, the following types of breast cancer are identified (i) Ductal carinoma in situ(DCIS):It is non invasive where the cancer cells are confined only to the ducts of the breast, (ii) Lobular carinoma in situ(LCIS): A non invasive condition where the cancer cells are limited to the milk producing glands of the tissue, (iii) Invasive ductal carcinoma(IDC): This is invasive type of cancer. The cancer cells develop in the milk ducts of the breast and then invade to the nearby tissues and organs. IDC is the most common type of breast cancer and (iv) Invasive lobular carinoma(ILC): This type of cancer begins in the lobules of the breast and spreads to the surrounding tissue.
Based on the size of the tumor and the extent of its spread into nearby tissues and organs the doctors recognize the different stages of cancer as stage 1, stage 2, stage 3 and stage 4 breast cancer. Stage 1 is a primary tumor with its size around 2cms while stage 4 is advanced where the tumor can have any size and the cancer cells have spread to nearby tissues and organs. Early detection of the cancer i.e., IOP Publishing doi: 10.1088/1757-899X/1065/1/012049 2 in the initial stages improves possibility of survival. If the cancer reaches stage 4, the chances of survival comes down. Thus early detection of cancer is very crucial and regular checks and screening can aid in detecting the symptoms earlier. Clinical experts diagnose the cancer after routine screening. The use of biomarkers and imaging tests helps the clinicians to identify the breast cancer. The most widely used image screening methods are Mammogram, ultrasound and Magnetic resonance imaging(MRI). Further researchers are trying to find new breast imaging methods to improve the detection and treatment of the cancer and have come up with Scintimammagraphy(molecular breast imaging), Positron emission mammography(PEM), Electrical impedance imaging(EIT) and Elastography Though imaging methods play an important role in decision making followed by treatment. They require expertise to deliver significant and accurate information . For example, extraction of feature descriptors from images using mathematical techniques to detect and differentiate the tumor and non tumorous region. Also the reliability of the detection method needs to be assessed based on (i) Accuracy, (ii) Precision, (iii) Correlation to the disease and (iv) Usability. Biomarkers [1], on the other hand are gaining lot of importance in accurately detecting and diagnosing the cancer as a result of advancements in bioinformatics tools. Recognition of significant biochemical changes in cancer has facilitated towards advances in detection and its treatment. They can be relied upon for early detection and accurate evaluation. Breast and prostate cancers are diagnosed comparatively earlier as compared to other cancers. Additionally, the cancer biomarkers are significant in detecting the disease for asymptomatic persons at the early stage. Ideally a portion of a protein or a protein in the blood or urine samples is considered as a marker for tumor detection. Thus this paper proposes a pattern recognition system that use biomarkers as features or descriptors combined with machine learning algorithms to effectively identify the presence or absence of cancer. Accordingly, this is a classification task where the classifier predicts the class label of the sample. The performance of the model is later quantitatively evaluated using the metrics derived from the confusion matrix.
The rest of the paper is organized as follows, Section 2 presents a comprehension of related work. Section 3 describes the methodology of the works. The experimental results and discussion are provided in section 4. Finally, section 5 concludes the paper.

Related work
Biomarkers in cancer are the characteristics that are measured and they describe the pathological state of an living being. Cancer biomarkers are categorized into predictive, diagnostic and prognostic based on how they are handled. The first cancer biomarker was recognized from the blood sample of a patient with colon cancer [2]. Later different biomarkers were developed to identify other cancers. The standards and measurements of breast cancer biomarkers are provided in [3] and a overview is found in [2].Further, the advancements in technology and data analytics is seen to solve various health related issues to improve the quality of health care and provide effective treatment. Recent years have witnessed a biggest transformation of Artificial intelligence in health care domain powered by machine learning algorithms that can assist the clinicians/experts in diagnosing the disease and build up effectual solutions. Thus the acquired dataset is fed to the algorithms that can discover the patterns and anomalies to make right decisions about the disease diagnosis.
Many researchers have put forward their ideas in using cancer biomarkers with machine learning algorithms to improve the precision of detection and analysis. Li and Chen [4] investigated the adaption of cancer attributes in detecting the disease which can reduce the probability of death rate. They used Wisconsin Breast cancer and Breast Cancer Coimbra datasets with Decision tree(DT), Random forest(RF), Support Vector Machine(SVM), Neural Network(NN) and Logistics Regression(LR) Classifiers and compared their performances . The experimental study showed that the Random forest algorithm provided better results in terms of prediction accuracy, F -score and Area Under the Curve(AUC) as compared to other classifiers. Farah Sardouk et al., [5] studied the predictors of Breast Cancer Coimbra dataset to estimate the presence or absence of the cancer. They created six machine models using Adaboost, Regression , Random Forest, Jrip, RBFNN and J48 classifiers in WEKA and MATLAB environments. The models were assesses using True positive rate, False positive rate, precision, recall, F1-score MCC and AUC. The results proved that the machine IOP Publishing doi:10.1088/1757-899X/1065/1/012049 3 learning algorithms were efficient in predicting the outcome. Miguel Patricio et al., [6] at university of Coimbra collected clinical attributes from 166 individuals based on anthropometric data. The measured data or features were then fed to logistic regression, Support vector machine and random forest machine learning algorithms to evaluate the probability of using the anthropometric data as biomarkers for cancer prediction. which were given to the developed a model that can be used as a biomarker. The evaluation of the classifier models for the metrics sensitivity, AUC and specificity indicated the better performance of SVM for the following predictors: Glucose, Resistin, BMI and age. Thus they can be efficient biomarkers of breast cancer.
Jin Yue et al., [7] carried out statistical analysis, spectral analysis and least squares fitting analysis on the Breast Cancer Coimbra dataset to check the interclass variation. The analysis was able to clearly indicate the significant distinction between the two classes: cancerous and non cancerous. Yolanda D. Austria et al., [8] investigated 11 classification algorithms and their variants : LR, k Nearest Neighbor(kNN), SVM, DT, RF, Gradient boosting and naive Baye's for predicting the breast cancer on the Coimbra dataset. The performance of the models were evaluated using accuracy and prediction time. The evaluation displayed an higher accuracy of 74.14% with Gradient boosting classifier and non linear SVM was fast in predicting the label without any delay. The work also identified the anthropometric data body mass index(BMI) as the principal predictor. Ratula Ray et al., [9] explored the use of DT, kNN, RF and Guassian naive Baye's machine learning algorithms in aiding the pathologists to make precise decision about the presence of the disease considering both image and numeric(biomarkers) databases. Authors used breast histopathology images from Kaggle and breast Coimbra dataset for the evaluation of classifier models. The analysis indicated the better performance of random forest classifier which provided precision , recall and F1 score of 71%, 71% and 70% respectively for the numeric dataset. Similarly it presented precision , recall and F1 score of 91%, 92% and 92% respectively. The study also implies the importance of resistin and BMI biomarkers that influence the decision process.
From the evaluation of the contributions it is found that majority of the methods used various machine learning algorithms independently to improve the success rate of cancer detection. This paper presents a model that makes use of a ensemble classification where the classifiers are fused at the decision level to enhance the performance of the system.

Methodology
The methodology of the proposed system to predict the presence of breast cancer is shown in Figure1.The frame work includes (i) Feature extraction (ii) Classifier/Machine learning model creation (iii) Classification. The extraction of suitable features or descriptors are essential in deciding the accuracy of the system. The anthropometric data measured from the input samples or the biomarkers form the features. These features are given to different classifiers for models creation which can predict the class label of the given sample. To enhance the presentation of the system, the decisions of the individual classifiers are combined based on decision level fusion strategy. The created models are evaluated to validate their performance. The details of the process is described in subsequent sub sections.

Dataset and Feature extraction
The work considered Breast Cancer Coimbra dataset [6] collected from 116 persons. From each participant nine clinical features were measured as displayed in Table 1.  Figure 2 identifies the distinction between the cancerous and non-cancerous samples that clearly indicate that they are potential biomarkers for the detection of the pathology .  which is provided to the classifiers for training and testing. To validate the proposed system, the dataset with its features was divided into two non overlapping training and testing sets with a ratio of 75:25. The training set is labeled using the class labels y={'1' , '0'}, where '1' indicates healthy and '0' indicates Cancerous. The labeled training set {FV,y} is passed to the classifier for creating the model.
The model created is then tested using the samples from the testing set to predict its outcome C ϵ y.

Machine Learning Algorithms
The classifier predicts the outcome of an unseen sample 'Q' as a function of labeled training set and the learning parameters of the algorithm .

Naive Baye's classifier
It is a simple probabilistic classifier which is based on Baye's theorom [10]. It requires fewer number of training samples for prediction. Baye's rule can be formulated as P(y/X)= (2) where X signify the features and y is the label representing the category of the input sample. In the context of the work presented here, the above equation can be rewritten by expanding the feature set as, Here C can be one of the class label belonging to y. Thus the outcome C is found based on maximum probability, i.e., C=argmaxC P(C) The classifier finds its applications in biometrics(face recognition), healthcare, weather prediction, security, emotion recognition, sentiment analysis etc.

Radial Basis Function Neural network(RBFNN)
RBFNN [11] is one of the variants of feed forward neural network that uses radial basis functions as the activation function in the hidden layer. The network converges faster to reach the performance goal. The NN architecture has three layers: Input layer, Hidden layer and Output layer. For a given input FV, the output at a node of output layer is computed as, (6) Where, FV is the input to the network/Feature vector is the weight from hidden neuron j to output neuron i ||.|| L2 / Euclidean norm is the RBF in the hidden neuron j with center Mj.

Linear Discriminant Analysis(LDA)
LDA classifier proposed by Fisher [12]is based on the concept of reducing the within(intra) class variation and maximizing the between(inter) class variation. This classifier finds class separation by minimizing the spread within class and maximizing the distance between each class. Thus it is based on the statistical measures mean and variance. For the given dataset and class labels, LDA works by projecting it on to hyperplane to find the scalar, of all the possible lines, the line that gives maximum class separability is chosen. The best projection vector is selected by maximizing the Fisher's criteria given by, J (W) = (7) here J(W) indicates the measure of difference between means of the classes SB represents the within class scatter matrix SW represents between class scatter matrix W is the projection matrix.

Results and Discussion
The experiment was conducted to identify the presence of the breast cancer for the given sample based on fusion of classifiers. The proposed method is validated with hold out method where the dataset with 116 samples was separated into training and testing sets with the ratio of 3:1. Thus the training set has 87 samples and testing set 29 samples. As a first step, all the training samples were passed to feature extractor that provided the feature vector FV with 9 dimensions. Each feature of the FV has different ranges, so the data is normalized using Z-score normalization that gives normalized FV with zero mean and standard deviation of one as shown FVn= (9) Later the FVn was labeled to form training data set Zi= . The training data set was accepted by the Naive Baye's(NB), Radial Basis Function Neural network(RBFNN) and Linear Discriminant Analysis(LDA) classifiers for creating the models. During training, the learnable parameters of the classifiers were modified based on the performance function. The RBFNN was optimized for a spread constant of 3.5 and for NB classifier the prior probabilities was set to [0.2 0.2]. The trained models were tested with the testing data set ZiQ= , where Q represents the Query or unseen sample.
The results of the individual classifiers were observed and analyzed using the parameters Receiver Operating Characteristics(ROC), Classification accuracy(CA), Precision, Recall(Sensitivity), Specificity and F1 score which are computed using the elements of confusion matrix [13] as shown, The ROC obtained for all the classifiers is displayed in Figure 3 and other metrics are shown in Table  2.
The analysis of ROC and the performance measures indicate the poor presentation of the classifiers when used individually. The highest CA, precision and specificity was obtained from RBFNN that provided 69%, 70% and 81.25% respectively.  To improve the performance of the system, the decision of the individual classifiers can be fused [14] which is applied in the proposed frame work. A review of the literature finds different fusion strategies based on max, min, sum, product, mean, median and majority vote rules. This work utilizes binary classifiers with class label vector y={'1', '0'}. Let C1, C2 and C3 be the outputs of NB, RBFNN and LDA classifiers such that Ci ϵ y. The outputs are fused as shown, C= F(C1, C2, C3) (15) here F signifies the fusion rule Now based on the fusion, we arrive at the final decision by opting for majority voting rule which requires odd number of single classifiers. This is the simple and a sensitive approach that assigns the sample with a most frequent class labels. Finally to demonstrate the significance of ensemble classification, the final decision was analyzed using the performance metrics and the results are tabulated in Table 1. The results show an 75% precision with ensemble classification that represents a low false positive rate . The analysis of F1-score which is the harmonic mean of precision and recall is advantageous in our work as the dataset considered has uneven class distribution(52 healthy and 64 cancerous samples). The range of F1-score is [0 1]. Thus a value of 0.7199 is better as compared to individual classifier results.
It is observed from the results that, the ensemble classification provided the results greater than the average of the single classifiers indicating a enhanced performance of the system subjected to CA, precision, recall, specificity and F1-score.

Conclusion
This paper proposes a pattern recognition system with ensemble of classifiers for classifying a sample into two classes viz presence of breast cancer and healthy. The work utilized anthropometric data /biomarkers as the features for data representation . These features or descriptors were provided to the NB, RBFNN and LDA machine learning algorithms for classifier model creation and evaluation. The experimental study on breast cancer Coimbra dataset with the proposed framework revealed the efficacy of the biomarkers as better features. Further the analysis of results obtained from fusing the decisions of single classifiers indicated improved performance measures in terms of classification accuracy, precision, recall, specificity and F1-score. This justified the deployment of ensemble classifiers in better discrimination of classes leading to enhanced performance of the system