{'fraction_non_alphanumeric': 0.03991958644457209, 'fraction_numerical': 0.023836875358989085, 'mean_word_length': 5.0454282407407405, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Machine Learning Algorithm Guiding Local Treatment Decisions to Reduce Pain for Lung Cancer Patients with Bone Metastases, a Prospective Cohort Study

Introduction As life expectancy increases for lung cancer patients with bone metastases, the need for personalized local treatment to reduce pain is expanding. Methods Patients were treated by a multidisciplinary team (MDT), and local treatment including surgery, percutaneous osteoplasty, or radiation. Visual analog scale (VAS) and quality of life (QoL) scores were analyzed. VAS at 12 weeks after treatment was the main outcome. We developed and tested machine learning models to predict which patients should receive local treatment. Model discrimination was evaluated by the area under curve (AUC), and the best model was used for prospective decision-making accuracy validation. Results Under the direction of MDT, 161 patients in the training set, 32 patients in the test set, and 36 patients in the validation set underwent local treatment. VAS in surgery, percutaneous osteoplasty, and radiation groups decreased significantly to 4.78 ± 1.28, 4.37 ± 1.36, and 5.39 ± 1.31 at 12 weeks, respectively (p < 0.05), with no significant differences among the three datasets, and improved QoL was also observed (p < 0.05). A decision tree (DT) model that included VAS, bone metastases character, Frankel classification, Mirels score, age, driver gene, aldehyde dehydrogenase 2, and enolase 1 expression had a best AUC in predicting whether patients would receive local treatment of 0.92 (95% CI 0.89–0.94) in the training set, 0.85 (95% CI 0.77–0.94) in the test set, and 0.88 (95% CI 0.81–0.96) in the validation set. Conclusion Local treatment provided significant pain relief and improved QoL. There were no significant differences in reducing pain and improving QoL among training, test, and validation sets. The DT model was best at determining whether patients should receive local treatment. Our machine learning model can help guide clinicians to make local treatment decisions to reduce pain. Trial registration Trial registration number ChiCRT-ROC-16009501.


INTRODUCTION
Bone metastases develop in 36% of patients with advanced lung cancers [1]. Bone metastases can lead to skeletal-related events (SREs) such as severe pain, pathologic fracture (PF), spinal cord compression (SCC), required radiation, bone surgery, and hypercalcemia [2], thereby significantly reducing the quality of life (QoL) of patients with lung cancer [3].
Although systemic medical treatments can control lung tumor growth [2], they are not sufficient to reduce pain, restore the integrity of bones, and allow a return to light weight-bearing [4]. Recent progression in lung cancer treatments, such as development of moleculartargeted agents, has improved patient survival [5]. With increasing life expectancy, there is growing need for effective local treatment for bone metastases to reduce pain and improve QoL. Surgery can restore the integrity of bones, but the decision of whether to perform surgery can be difficult, as the risks may outweigh benefits of pain reduction and improved function [6]. Alternatively, percutaneous osteoplasty (POP) is an effective and safe palliative therapy to reduce pain and improve QoL [7]. Further, radiation can improve reduced QoL caused by painful bone metastases [8]. In addition, previous work has emphasized the requirement of a multidisciplinary team (MDT) approach involving a team of specialists in oncology to reduce pain and improve patients' QoL [9].
Local treatment indications have been controversial, and local treatment for lung cancer patients with bone metastases must be individually tailored to each patient with consideration for multiple factors. As there is increasing need for automatic and accurate analysis for clinical use, machine learning offers a solution to generate reasonable generalizations, discover patterns, and enable more accurate decisionmaking [10]. Machine learning may facilitate more effective assessments by physicians [11]. We previously built machine learning models to predict SREs risk [12]. The purpose of this study was (1) to evaluate the feasibility and effectiveness of applying our local treatment algorithm to reduce chronic pain and improve patients' QoL, and (2) to develop and validate machine learning models for local treatment decisionmaking to reduce pain in lung cancer patients with bone metastases.

Patient Selection
This study was approved by the ethics committee of Shanghai Sixth People's Hospital Affiliated to Shanghai Jiaotong University in October 20, 2016 and was registered in the Chinese Clinical Trial Registry (No. ChiCRT-ROC-16009501) in October 20, 2016. Principles of the Declaration of Helsinki were followed. Written informed consent was obtained from all participants in this study.
Inclusion criteria were: (1) age over 18 years, (2) pathology-proven diagnosis of lung cancer and radiographical/pathological evidence of bone metastases, (3) no previous treatment for bone metastases, and (4) good general condition, as measured by Eastern Cooperative Oncology Group (ECOG) performance scores of 0-2 with an estimated survival time of more than 3 months.

Management Algorithm
Indications for systemic treatments, including chemotherapy, target therapy, and bone-targeting agents (BTAs), as well as local treatment, including surgery, POP, and radiation, were evaluated by an MDT of medical oncologists, radiation oncologists, interventional radiologists, orthopedic oncologists, and pain specialists. The algorithm is shown in Fig. 1.
Spinal stability was ascertained using Spinal Instability Neoplastic Scores (SINS) [13], and the risk of pathological fracture for the appendicular skeleton was ascertained using the Mirels scoring system [14]. Surgical procedures followed guidelines of the Global Spine Tumor Study Group and Italian Orthopedic Society [15][16][17][18]. Procedures for POP were introduced by our MDT in 2012 [7]. Radiation was performed mainly with 6-MV photons using linear accelerators. Dose fractionation schedules included multi-fraction radiation, such as 30 Gy in ten fractions. Adjuvant therapy-like radiation [19] was used after surgery or POP to prevent tumor recurrence.
Informed consent was obtained for all patients in the study. If local treatment was performed, informed consent by the patient or a legal guardian was obtained 24 h before initiation and after thorough explanation of the methods, potential complications, and alternative treatments.
Patients were asked to complete a questionnaire that assessed severity of pain using the mean daily visual analog scale (VAS) [21] and QoL using the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Bone Metastases Module (EORTC QLQ-BM22) [22,23] 1 day before and at 1, 6, 12, and 24 weeks after local treatment or enrollment. VAS at 12 weeks was the main outcome. Patients were followed for survival every 3 months.

Cost Valuation
Cost analyses of individual patients were estimated from a payer perspective using health resource utilization data from patient charts. Costs of procedures performed during an inpatient stay were assumed to be captured in diagnosis-related group costs.

Model Development
Decision trees (DT) (eXtreme Gradient Boosting, XGBoots), support vector machine (SVM), and Bayesian neural networks (BNN) were used to build local treatment decision-making models. On the basis of our previous research [12], we selected the following predictor variables: sex, age, ECOG, VAS, bone metastases character, extent of bone metastases, visceral metastases, Frankel classification, and Mirels scale. As additional predictor variables, we selected lung cancer pathology, lung cancer driver gene, and five differentially expressed bone metastasis proteins. The model output was 0, no local treatment; or 1, local treatment. The training set including patients who had improved VAS and QoL measures after local treatment.

Model Performance
In the training set, we used tenfold cross validation. The test set consisted of data not associated with the training set. Model discrimination was evaluated by area under the receiver operator characteristic curves (AUC). Sensitivity, specificity, and accuracy were used to evaluate model performance.

Model Validation
The validation subset was used to prospectively evaluate the accuracy of DT models in predicting whether patients would receive local treatment. DT models made the decision whether to take local treatment or not. The MDT made the final decision about which local treatment to provide to patients and could reject the model's decision. Compared with MDT, the AUC, sensitivity, specificity, and accuracy were used to evaluate model performance.

Statistical Analysis
Stata Corp 2013 (Stata Statistical Software: Release 13; StataCorp LP, College Station, TX, USA) and Python Version 3.6 (Python Software Foundation, Wilmington, DE, USA) were used to analyze data and build the model. Median values and ranges were determined for descriptive statistics. Chi-square and Fisher's exact tests were used for categorical variables. Student's t tests and Mann-Whitney tests were used for continuous and ordinal variables. Wilcoxon signed-rank tests were used to compare paired outcomes at various follow-up times. The Kaplan-Meier method was used to estimate survival. A p \ 0.05 was considered statistically significant.  Table 1. Treatments in training, test, and validation sets are shown in Fig. 2b-d.

Post-treatment Pain
VAS scores before treatment for all 746 patients in surgery, POP, radiation, and no local treatment groups were 5.70 ± 1.22, 5.53 ± 1.34, 6.62 ± 1.48, and 3.37 ± 1.38, respectively; scores were highest in the radiation group and lowest in the no local treatment group (p \ 0.05). VAS scores in surgery, POP, and radiation groups decreased significantly to 4.78 ± 1.28, 4.37 ± 1.36, and 5.39 ± 1.31, respectively, at 12 weeks after local treatment (p \ 0.05). VAS scores for patients in the no local treatment group did not significantly differ 12 weeks after enrollment. Detailed scores are shown in Fig. 3a.
VAS scores in training, test, and validation sets all decreased significantly at 12 weeks after surgery, POP, or radiation (p \ 0.05), with no significant differences among the three sets. Detailed scores are shown in Fig. 3b-d.

Post-treatment QoL
Pain sites (PS) and pain characteristic (PC) scores of the QLQ-BM22 before treatment for all 746 patients were highest in the radiation group and lowest in the no local treatment group (p \ 0.05), while functional interference (FI) and psychosocial aspects (PA) scores were highest in the no local treatment group and lowest in the radiation group (p \ 0.05). Patients had improved QoL scores 12 weeks after surgery, POP, or radiation (p \ 0.05). PS and PC scores decreased significantly while FI and PA scores increased significantly at 12 weeks after local treatment in surgery, POP, and radiation groups (p \ 0.05). PS, PC, FI, and PA scores for patients in the no local treatment group did not significantly differ 12 weeks after enrollment. Pre-treatment and post-treatment subscores in pain and functional domains in QLQ-BM22 are shown in Fig. 3a.
In training, test, and validation sets, PS and PC scores decreased significantly while FI and PA scores increased significantly at 12 weeks after surgery, POP, or radiation (p \ 0.05), with no significant differences among the three sets. Detailed scores are shown in Fig. 3b-d.

Cost Valuation
Mean costs during 24 weeks for all 746 patients in surgery, POP, radiation, and no local treatment groups were $21,172 ± 8626, $16,142 ± 5078, $15,899 ± 5527, and $13,526 ± 5685, respectively; costs were highest in the surgery group and lowest in the no local treatment group (p \ 0.05). There were no significant differences in mean costs among training, test, and validation sets in the four treatment groups. Detailed costs are shown in Fig. 4a and b.

Survival
Median follow-up was 15 months (range 6-41 months). The endpoint of analyses was overall survival time (OS), and a total of 548 patients died. The OS was 18.03 ± 0.45 months in all 746 patients, and the 1-year survival rate was 65.55%. OS was 15.50 ± 1.08, 16.72 ± 1.04, 16.90 ± 1.08, and 18.25 ± 0.54 months in surgery, POP, radiation, and no local treatment groups, respectively, with no significant differences. One-year survival rates were 61.11%, 60.68%, 63.38%, and 66.73% in surgery, POP, radiation, and no local treatment groups, respectively, with no significant differences. OS did not significantly differ among the three datasets.

Model Development and Validation
The DT model included VAS, bone metastases character, Frankel classification, Mirels

DISCUSSION
Management of bone metastases has been considered palliative and not associated with patient prognosis and thus has not been given much importance in the past. Recently, however, it has become necessary to initiate bone management programs concurrently with cancer treatment to effectively reduce pain and improve patients' QoL. Similar to previous studies, surgery, POP, or radiation in this study provided significant pain relief and improved QoL [6,8,[25][26][27][28]. We found that pain-related scores can be relieved quickly, but functional relief was slower than pain relief, especially in patients who underwent surgery and radiation. Surgical patients need to recover from postoperative trauma, and radiation has a slower effect on the recovery of mechanical stability. However, we found that the pain increased after 6 months after treatment, which was related to the characteristics of bone metastasis. Some patients underwent secondary local treatment. Breakthrough cancer pain (BTcP) is defined as ''a transitory flare of pain that occurs on a background of relatively well-controlled baseline pain'' [29]. It occurs in patients with cancer regardless of chronic pain management. Typical BTcP episodes are of moderate to severe intensity, rapid in onset (min), and of relatively short duration (median 30 min). In our study, local treatment was good at controlling chronic pain, but can not be rapid in onset of action.
In this study, we found that our MDT algorithm was effective in providing treatment decisions that provided significant pain relief and improved QoL. The mechanical stability, neurological risk, oncological parameters, and preferred treatment (MNOP) algorithm [30] suggests surgery or radiation as the main treatment for spinal metastases. Minimally invasive approaches such as POP are also recommended *Pain level on a 10-point scale, with 0 representing no pain and 10 representing maximum pain intensity imaginable. Grade 1, 0-3; grade 2, 4-6; grade 3, 7-10 **Visceral metastases defined as distant metastases, except for BM, including brain metastases ***Frankel classification defined as 0, without spine metastasis; 1, A; 2, B; 3, C; 4, D; 5, E ****Mirels scale defined as 0, without extremity metastasis; 1, 4-6; 2, 7-9; 3, 10-12 *****Pathology defined as 1, adenocarcinoma; 2, squamous cell carcinoma; 3, poorly differentiated cancer; 4, large cell carcinoma; 5, small cell carcinoma by our algorithm. We have treated hundreds of spinal PF and instabilities with POP instead of high-risk surgery. We find that patients recover spinal stability after POP with low morbidity, and our results here show that mean costs for spine metastases in the POP group were much lower than those in the surgery group. However, we note that POP does not easily restore structural integrity and weight-bearing for the appendicular skeleton and that surgery can quickly restore these functions with less risk. To prevent PF, we have used preventive surgery or POP, and our algorithm shows that surgery and POP are complementary. We prefer POP for spinal metastases and surgery for appendicular skeleton metastases.
Our results show that local treatment did not negatively influence OS for lung cancer patients with bone metastases. According to some studies, patients surgically treated for bone metastases survive for less than 10 months [26]. Recently, Tang et al. reported OS of 14 months in lung cancer patients with spinal metastases, and patients who underwent surgery had longer survival [27]. However, our study excluded patients with a life expectancy of less than 3 months, 92.6% patients received systemic medical treatments (57.9% for targeted agents), and Frankel classification in the no local treatment group was almost E, while in Tang's study it was A-D, possibly accounting for the longer survival in our study.
Machine learning models can help guide treatment decisions. Although the MDT approach is an effective method to manage bone metastases, it can be difficult to manage patients who may develop serious SREs in a timely manner by holding weekly meetings.
Alternatively our machine learning models based on routinely available clinical parameters were constructed for local treatment decisionmaking. XGBoost is a novel boosting tree-based ensemble algorithm that has gained wide popularity in the machine learning community [31]. The DT model showed greater accuracy than SVM and BNN models, and it included driver genes and that ALDH2 and ENO1 expression had higher accuracy, which is in line with current needs for precision medicine. For ethical reasons, the final decision was still left to the MDT. However, pain, QoL, mean cost, and OS of the four treatment groups did not significantly differ among training, test, and validation sets. Feasibility and stability of the DT model were satisfactory, and the DT model was good at determining whether patients should receive local treatment in prospective clinical validation. Thus, the DT model may help clinicians decide on local treatment for individual patients to reduce pain and improve patients' QoL.
To the best of our knowledge, this is the first attempt to use machine learning techniques for local treatment decision-making models in lung cancer patients with bone metastases. However, the algorithm cannot completely solve the problem of patient classification, and some treatments have not been carried out in our study, which represents a weakness. For example, stereotactic body radiation therapy for painful spine metastasis shows better results in local control and pain relief than standard 2D or 3D techniques [30], and recent development of immune checkpoint inhibitors has fundamentally changed how patients with metastatic lung cancer are treated [5]. However, machine learning models could only guide decisions about whether to apply local treatment or not, and only the accuracy of the DT model in local treatment decision-making compared with MDT was evaluated. However, as more patients receive local treatment, thus increasing data availability, we will be able to develop additional models that can better guide types of local treatment. Further, this study is limited in that it involved a single center and the number of patients in the test and validation sets is small. A randomized controlled multicenter b Fig. 3  Compliance with Ethics Guidelines. The study was performed in accordance with ethical principles that have their origin in the Declaration of Helsinki and are consistent with International Conference on Harmonization (ICH)/Good Clinical Practice (GCP), applicable regulatory requirements, and the sponsor or its delegate's policy on bioethics. This study was approved by the ethics committee of Shanghai Sixth People's Hospital Affiliated to Shanghai Jiaotong University in October, 20, 2016 and the TRN is 2016-162. This study was registered in the Chinese Clinical Trial Registry (No. ChiCRT-ROC-16009501). Written informed consent was obtained from all participants in this study.
Data Availability. The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
Open Access. This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/bync/4.0/.