{'fraction_non_alphanumeric': 0.03176572775778633, 'fraction_numerical': 0.02097034371510113, 'mean_word_length': 4.939204126750184, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 1, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Application of Machine Learning Techniques to Predict Bone Metastasis in Patients with Prostate Cancer

Objective This study aimed to develop and validate a machine learning model for predicting bone metastases (BM) in prostate cancer (PCa) patients. Methods Demographic and clinicopathologic variables of PCa patients in the Surveillance, Epidemiology and End Results (SEER) database from 2010 to 2017 were retrospectively analyzed. We used six different machine learning algorithms, including Decision tree (DT), Random forest (RF), Multilayer Perceptron (MLP), Logistic regression (LR), Naive Bayes classifiers (NBC), and eXtreme gradient boosting (XGB), to build prediction models. External validation using data from 644 PCa patients of the First Affiliated Hospital of Nanchang University from 2010 to 2016. The performance of the models was evaluated using the area under receiver operating characteristic curve (AUC), accuracy score, sensitivity (recall rate) and specificity. A web predictor was developed based on the best performance model. Results A total of 207,137 PCa patients from SEER were included in this study. Of whom, 6725 (3.25%) developed BM. Gleason score, Prostate-specific antigen (PSA) value, T, N stage and age were found to be the risk factors of BM. The XGB model offered the best predictive performance among these 6 models (AUC: 0.962, accuracy: 0.884, sensitivity (recall rate): 0.906, and specificity: 0.879). An XGB model-based web predictor was developed to predict BM in PCa patients. Conclusion This study developed a machine learning model and a web predictor for predicting the risk of BM in PCa patients, which may help physicians make personalized clinical decisions and treatment strategy for patients.


Introduction
Prostate cancer (PCa) is the most common non-skin cancer among men globally, with approximately 1.6 million cases and 366,000 deaths reported each year. 1,2 Metastatic prostate cancer has important clinical implications, and metastatic disease may occur on the occasion of the initial clinical diagnosis. 3,4 Bone metastases (BM) take up a great proportion of patients with metastases, accounting for approximately 16.7% of all metastatic cases, and these patients had significantly reduced 5-year survival and quality of life. 5,6 due to the radiation injury and the high cost. 8,9 The current method for detecting BM in PCa patients is mainly bone scan. But it just suggested for patients with suspected skeletal-related events (SRE) due to the severe radiation injury. In addition, the median time to SRE has been reported to be 5 months after bone metastases. 10 An assisted decision-making system that can help determining which cancer patients should receive bone scan will provide help for addressing these issues. And with the development of precision medicine, the diagnosis and treatment of cancer should be individualized. The Surveillance, Epidemiology and End Results (SEER) database is a publicly obtainable, federally funded cancer reporting system that brings raw material to our insights into complex diseases. 11 With the increasing level of computer hardware, machine learning can facilitate the diagnosis of cancer metastasis by processing and analyzing large, heterogeneous and complex clinical data and building predictive models. And there are already studies that have been conducted in this area with good results. [12][13][14] Therefore, in this study, we aim to build a prediction model to evaluate the risk of BM in PCa patients based on machine learning techniques, and develop a web-based predictor that can be easily manipulated by physicians and patients. This study may provide some help for clinicians to make personalized decisions for the treatment of patients with PCa BM.

Study Population
The study is carried out based on the SEER database. Patients' data were obtained from the "SEER Research Plus Data, 18 Registries, Nov 2019 Sub (2000-2017)" and downloaded using SEER*stat 8.3.9 software. Patients who were diagnosed with PCa from 2010 to 2017 were included in this study. Exclusion criteria were as follows: 1 PCa was not the first tumor. 2 The information of race, grade, Prostate-specific antigen (PSA), Gleason score, T, N stage, metastatic status and marital status missed or unknown. Additionally, 644 PCa patients from the First Affiliated Hospital of Nanchang University from 2010 to 2016 were included for external validation. The case screening process is shown in Figure 1.

Data Selection
Eight variables from the SEER database that may affect BM in PCa patients were selected in this study, including age at diagnosis, race, grade, PSA value, Gleason score, T, N stage and marital status. Besides, we only included patients with PSA values between 0.1 and 98.0ng/mL. Because no specific values over 98.0 ng/mL were provided by the SEER database. All patients enrolled in this study were staged using the 7th edition of the AJCC TNM staging system and relevant guidelines of the SEER program.

Statistical Analyses and Model Establishment
All statistical analysis was performed in Python (version 3.8, Python Software Foundation) and SPSS (version 26, IBM, USA). 15 All machine learning algorithms were built based on scikit-learn (version 0.24.1). The patient data were randomly sliced into training and internal test sets in a ratio of 7:3 using python. Training set is used to build the model, and the internal test set is used for model validation and evaluation.
In order to determine the variables included in the machine learning model, we conducted a univariate analysis to compare these variables between patients with and without BM. The Chi-square test was utilized for categorical data, and the Wilcoxon rank-sum test was used for continuous non-normally distributed data. Variables with a P < 0.05 in univariate analysis were enclosed within the construction of machine learning models and multivariate logistic regression was performed to identify the risk factor for BM. Meanwhile, based on the Permutation Importance principle, 16 we performed feature importance analysis on the variables in each machine learning model. Correlation analysis was performed on the screened variables to test whether the variables would affect each other.
This study used six different machine learning algorithms to model the data, including Decision tree (DT), Random Forest (RF), Multilayer Perceptron (MLP), Logistic regression (LR), Naive Bayes classifiers (NBC) and eXtreme gradient boosting (XGB). [17][18][19][20][21][22] The ML algorithms were trained by using Python software to predict the BM in PCa patients. Model parameter settings are detailed in the Supplementary Materials. Parameter settings link is accessible from https://share.streamlit.io/liu wencaincu/prostate-cancer/main/prostate.py. Then, the predictive power of the machine learning models was evaluated in internal ten-fold cross-validation of the train set, internal test set and external test set. The area under the receiver operating characteristic curve (AUC), the https://doi.org/10.2147/CMAR.S330591

DovePress
Cancer Management and Research 2021:13 8724 sensitivity (recall rate), specificity, and accuracy score were calculated. The best-performing model was selected to build a web predictor.

Demographic Baseline Characteristics
A total of 207,137 PCa patients from SEER database were included in the study. Of whom, 6725 (3.25%) developed BM and 200,412 (96.75%) had no BM. All demographic and clinicopathological characteristics of these patients were demonstrated in detail in Table 1. All patients were randomly cut into a training set (n = 144,995) and an internal test set (n = 62,142) in a ratio of 7:3. External validation was conducted by using data of 644 PCa patients from the First Affiliated Hospital of Nanchang University. The details of the training and test sets are shown in Table 2.

Univariate and Multivariate Logistic Regression Analysis
Based on the univariate analysis, age, race, grade, T, N stage, PSA value, Gleason score and marital status were significantly associated with the BM in PCa patients (P < 0.05) ( Table 3). Variables with a P values <0.05 between these two groups were selected for multivariate logistic regression analysis. Based on the analysis, T, N stage, Gleason score and PSA value were found to be the independent risk factors for BM in PCa patients (Table 3).

Variable Influence and Feature Importance on Prediction
Correlation tests were performed among different variables identified from the univariate analysis. The correlation heat map showed that variables were not significantly correlated with each other (Figure 2), which indicated that they were mutually independent.

8726
The importance of features in each machine learning model for predicting BM is shown in Figure 3. Although the importance of features varied slightly among different machine learning algorithms, PSA value, Gleason score and N stage ranked at the top three of five models, which had similarities with the results of multivariate logistic analysis. In contrast, marital status ranked at the last place in most algorithms, but it also made some contributions to the models. In the XGB model, the importance of features were sorted in descending order by PSA value, Gleason score, N stage, T stage, age, grade, race and marital status.

Model Performance
The predictive performance of different models was compared using the internal ten-fold cross-validation of training set, internal and external test sets, which were detailed in Figures 4-6 and Table 4. Among these models, the XGB model showed the best performance with an average AUC of 0.951 in the internal ten-fold cross-validation ( Figure 4). In the internal test set, the XGB model gained the best score with an AUC of 0.955, an accuracy of 0.881, a sensitivity (recall rate) of 0.905 and a specificity of 0.880. In the external test set, the XGB model also showed excellent performance with an AUC of 0.962, an accuracy of 0.884, a sensitivity (recall rate) of 0.906 and a specificity of 0.879 ( Figures 5 and 6 and Table 4). Meanwhile, the prediction results of different models were presented with a heat map in Figure 7.

Web Predictor
A web predictor based on the best predictive performance of the XGB model was developed to predict BM in PCa patients. The risk of BM from PCa could be easily predicted by simply setting the variables in the sidebar of the web page (https://share.streamlit.io/liuwencaincu/prostatecancer/main/prostate.py) (Figure 8).

Discussion
Prostate cancer is the second leading cause of cancerrelated death among men in the world. Bone-related events usually occur after bone metastases, which will result in reduced patient quality of life and survivorship. 23 The current diagnostic method for BM from PCa is bone scan and prostate-specific membrane antigen examination combined with nuclear imaging. However, there is radiation injury and high cost for the bone scan, and not all PCa patients are recommended for BM screening. [24][25][26] Tissue biopsy is another method for the diagnosis. But it may increase the risk of further tumor invasion. Although skeletal-related events are considered to be a sign of BM, it would not be reasonable for screening BM from PCa because it may delay the treatment. Thus, it makes sense to develop With the development of the computer technique, machine learning technology has been widely used in different fields. And it also shows great promise for application in the biomedical science. 13,27 Studies have already used machine learning technology to predict the development of diseases. 12,28 In this study, several widely used machine learning algorithms were developed and validated to predict the risk of BM in PCa patients. After the comparison of algorithms with several evaluation indicators, the XGB algorithm-based prediction model showed the best performance among these models. The XGB model achieved better performance than others probably because it uses a number of strategies to prevent overfitting, exploits the secondorder derivatives of the loss function and supports parallelization, and has a fast data processing speed. 29 These results can provide clinicians with more accurate prediction outcomes and help them to make personalized decisions for the treatment of PCa.
In the present study, four risk factors, including T, N stage, PSA and Gleason Score, were screened out of the pre-selected eight factors by univariate and multivariate analyses. They were in high agreement with the models' feature importance ranking (Table 4, Figure 3). However, machine learning algorithms consider that other variables that are not statistically significant can also make some contribution to the prediction. This may be because the algorithms can make predictions by exploring intrinsic connections between data that cannot be discovered through traditional statistical methods.
Studies have found that PCa patients with high PSA values have a higher likelihood of developing BM. It is recommended that PCa patients with PSA >20ng/mL need a bone scan to check for BM. 30,31 In this study, we also found that PSA values were important in predicting BM from PCa. It has been shown that advanced age and clinical stage were both correlated with the BM from PCa and poor prognosis. 32,33 Chen et al 34 found that age greater than 70 years was the threshold for significantly higher risk of BM for PCa patients. However, Stolzenbach et al 35 found that age and race were not informative in predicting the progression of PCa metastasis, which was in  32,36 Patients with regional lymphoma metastases tend to be more likely to develop BM than those with N0 stage. 37 The Gleason score also played an important role in the prediction model. The degree of BM in patients varies greatly with

8731
the change of Gleason score. 38 It was reported that Gleason score greater than 6 had a high specificity (88.9%) for the diagnosis of BM. 39 In addition, unmarried male patients had a higher risk of BM from PCa, which was in line with previous studies. 36,40 Our model adequately incorporates various risk factors that may affect BM in PCa patients and achieves excellent predictive performance. Based on machine learning algorithms and the huge amount of data in the SEER database, a model was constructed to predict BM in PCa patients and a web page predictor was developed. However, there are still some limitations in this study. First, the population in this study was obtained from the SEER database and externally validated using data just from a single center, which will be limitations for the application. Second, due to the inherent black-box properties of machine learning algorithms, it may pose some difficulties for the interpretation of the model. Third, the SEER database just reports the initial diagnostic information of PCa patients, and  further therapeutic information is missing. We cannot access this information for further analysis.
In conclusion, we developed a prediction model to predict the risk of BM in PCa patients based on the XGB algorithm with machine learning techniques, and developed a web predictor in this study. And those at high risk of BM were recommended for further detailed screening based on the web predictor. This may help physicians to individualize the treatment of BM in patients with PCa.

Data Sharing Statement
The datasets generated and/or analyzed during the current study are available in the SEER database (https://seer. cancer.gov/).

Ethics Approval and Consent to Participate
We received permission to access the research data file in the SEER program from the National Cancer Institute, US. Approval was waived by the local ethics committee, as SEER data is publicly available and de-identified. This study was approved by the Ethics Committee of the First Affiliated Hospital of Nanchang University, and cases from the First Affiliated Hospital of Nanchang University signed written informed consent form. This study followed the guidelines outlined in the Declaration of Helsinki.