{'fraction_non_alphanumeric': 0.0384719222462203, 'fraction_numerical': 0.021058315334773217, 'mean_word_length': 5.065775109170306, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 2, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A nomogram prognostic model for large cell lung cancer: analysis from the Surveillance, Epidemiology and End Results Database

Background Currently, there is no reliable method for predicting the prognosis of patients with large cell lung cancer (LCLC). The aim of this study was to develop and validate a nomogram model for accurately predicting the prognosis of patients with LCLC. Methods LCLC patients, diagnosed from 2007 to 2009, were identified from the Surveillance, Epidemiology and End Results (SEER) database and used as the training dataset. Significant clinicopathologic variables (P<0.05) in a multivariate Cox regression were selected to build the nomogram. The performance of the nomogram model was evaluated by the concordance index (C-index), the area under the curve (AUC), and internal calibration. LCLC patients diagnosed from 2010 to 2016 in the SEER database were selected as a testing dataset for external validation. The nomogram model was also compared with the currently used American Joint Committee on Cancer (AJCC) tumor-node-metastasis (TNM) staging system (8th edition) by using C-index and a decision curve analysis. Results Eight variables—age, sex, race, marital status, T stage, N stage, M stage, and treatment strategy—were statistically significant in the multivariate Cox model and were selected to develop the nomogram model. This model exhibited excellent predictive performance. The C-index and AUC value were 0.761 [95% confidence interval (CI), 0.754 to 0.768] and 0.886 for the training dataset and 0.773 (95% CI, 0.765 to 0.781) and 0.876 for the testing dataset, respectively. This model also predicted three-year and five-year lung cancer-specific survival (LCSS) in both datasets with good fidelity. This nomogram model performs significantly better than the 8th edition AJCC TNM staging system, with a higher C-index (P<0.001) and better net benefits in predicting LCSS in LCLC patients. Conclusions We developed and validated a prognostic nomogram model for predicting 3- and 5-year LCSS in LCLC patients with good discrimination and calibration abilities. The nomogram may be useful in assisting clinicians to make individualized decisions for appropriate treatment in LCLC.

the general condition of the patient (3,4). Previous studies have reported several clinicopathological factors that affect the prognosis of LCLC, including the LCLC subtype, tumor location, tumor stage, and others (5,6).
Nomogram is a graphical prognostic calculating tool for predicting the prognosis of cancer patients, which may facilitate better prognosis assessment and treatment stratification. Studies have shown that this user-friendly survival prediction tool can improve medical care in patients with gastric cancer, liver cancer, small cell lung cancer, and major salivary gland cancer (7)(8)(9)(10). However, to the best of our knowledge, no similar nomogram tool is available for predicting LCLC prognosis until now.
In September 2019, He et al. (11) published the only study of nomograms in large cell neuroendocrine carcinoma (LCNEC). In his study, he found that age, gender, tumor stage, N stage, size, and surgery of primary site were the independent prognostic factors of 3-and 5-year overall survival (OS) and developed a nomogram model. He's nomogram has some limitation. He only analyzed the impact of surgery on the prognosis of LCNEC patients but does not analyze the impact of the other two commonly used treatment methods (chemotherapy and radiotherapy) on the prognosis of LCNEC. In addition, LCNEC was not a subtype of LCLC in the 2015 World Health Organization (WHO) Lung Tumors Classification, which had been grouped with other neuroendocrine tumors (12).
The aim of this study was to collect information on LCLC patients from the Surveillance, Epidemiology and End Results (SEER) database, perform statistical analysis to identify prognostic clinicopathological factors and develop and validate a new nomogram model for LCLC patients. A reliable and high-performance nomogram model would be useful for identifying high-risk patients and planning appropriate adjuvant therapeutic strategies for LCLC patients.

Methods
This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The present study was approved by the Ethics Committee of the Peking University First Hospital (ethics number: 2018-236) and individual consent for this retrospective analysis was waived.

Study population
This retrospective study was conducted by acquiring data from the SEER database using SEER*STAT 8.3.5 software. A flow-chart illustrating the methodology used for identifying cases of LCLC in the SEER database during 2007-2016 ( Figure 1). Patients with LCLC were identified using International Classification of Diseases for Oncology (ICD-O) topography codes of "8012.2", "8012.3", and "8014.3". All patients enrolled between 2007 and 2016 were selected. Included patients met the following criteria: (I) patients with age ≥18 years; (II) patients with pathologically confirmed LCLC; (III) patients with only one primary tumor; (IV) patients with accurate follow-up information. Exclusion criteria: (I) patients with incomplete data; (II) patients with LCNEC. LCNEC was excluded because its biological, clinical, and prognostic characteristics in advanced stages were similar to those of SCLC. In the 2015 World Health Organization (WHO) Lung Tumors Classification, LCNEC was grouped with other neuroendocrine tumors and was no longer a subtype of LCLC (12). A total of 3,641 LCLC cases were identified from the SEER database, of which 2,971 patients met our inclusion criterion and were included in our study. These patients were divided into two groups based on the year of diagnosis: the training dataset (diagnosed from 2007 to 2009) and the testing dataset (diagnosed from 2010 to 2016). The nomogram model was developed from the 1,669 LCLC patients from the training dataset and validated by the 1,302 cases in the testing dataset.

Collected variables
Collected data consisted of clinicopathological, treatment, and follow-up information, including the year of diagnosis, sex, age, race, site of primary tumor, laterality, grade, chemotherapy, radiotherapy, marital status, the American Joint Committee on Cancer (AJCC) tumor-node-metastasis (TNM) stage, T category, N category, M category, tumor size, surgical status, survival status, overall survival (OS) time, and lung cancer-specific survival (LCSS) time. Age was a continuous variable, which was converted to a categorical variable in this study. The optimal cutoff values determined by X-tile (version 3.6.1, Yale University) software were 54-and 81-year, respectively. OS was defined as the interval from diagnosis to death or last follow-up, and LCSS was calculated from the date of diagnosis to the date of cancer-specific death. The AJCC TNM stage, T descriptor, N descriptor, and M descriptor were reclassified according to the 8 th edition lung cancer staging system defined by the AJCC (13). Treatment methods included surgery, chemotherapy, and radiation therapy, and different treatment methods were combined to form seven treatment strategies, including no treatment, surgery (S), chemotherapy (C), radiotherapy (R), surgery combined with chemotherapy (SC), surgery combined with radiotherapy (SR), chemotherapy combined with radiotherapy (CR), and surgery combined with chemoradiotherapy (SCR).

Establishing nomogram prognostic model
The nomogram was established using the training dataset. Clinicopathological variables, including age, sex, race, marital status, site of the primary tumor, tumor grade, laterality, T descriptor, N descriptor, M descriptor, and treatment strategy were statistically analyzed to select prognostic factors for LCSS. Multivariate Cox regression analysis was used to estimate the predictive factors and their weights. Akaike information criterion (AIC) was used to develop multivariate models by removing predictors that were less statistically significant starting from a full model containing all predictive variables (14).

Validation of model
The following methods were used to evaluate the predictive performance of this nomogram model. First, a risk score was calculated for each case in the training dataset based on the established Cox regression model. By quartile stratification of the risk score, the patients in the training dataset were divided into four groups, and Kaplan-Meier survival curves were plotted according to the level of the risk score. Second, bootstrapping (1,000 repetitions) was used to obtain a relatively unbiased estimate of the models' performance (8). The concordance index (C-index) and a calibration plot were used to determine the performance of the model in distinguishing between high-risk and low-risk patients. Third, the area under the curve (AUC) of timedependent receiver operating characteristics was calculated. Fourth, external validation using cases in the testing dataset was performed, the C-index was calculated, and AUC and the calibration curve were plotted. Finally, the performance of the nomogram model was compared to the AJCC TNM staging system (8 th edition) by using the C-index and decision curve analysis (15,16).

Statistical Methods
Continuous variables were converted to categorical variables with the median used as a cutoff value, and categorical variables were denoted as percentages. Differences between groups were evaluated with χ 2 tests. Survival outcome was calculated according to the Kaplan-Meier method, and logrank tests were used to determine significant differences between survival curves. All statistical analyses were conducted using SPSS 22.0 software (SPSS Inc., Chicago, IL, USA) and R version 3.6.0 (R Foundation for Statistical Computing, Vienna, Austria). R packages "foreign", "survival", "rms", "Hmisc", "rmda", "survivalROC", and "compareC" were used. The source codes of R software in this study were provided in Supplementary (Appendix 1). Results with P value <0.05 were considered statistically significant.

Clinicopathologic characteristics of patients with LCLC
The characteristics of the two datasets are shown in Table 1.
In comparing the training and testing datasets, T descriptor (P=0.031), M descriptor (P<0.001), and treatment strategy (P=0.047) are significantly different. Other variables, such as age, sex, race, marital status, site of the primary tumor, laterality, N descriptor, and surgical treatment are similar between groups. The median follow-up time is 6.9 months (range, 0 to 119 months) for the training dataset and 6.0 months (range, 0 to 83 months) for the testing dataset. The LCSS and OS in the training dataset are shown in Figure 2. The three-year LCSS and OS were 20.2% and 17.4%, respectively. Five-year LCSS and OS were 15.2% and 11.6%, respectively.

Independent prognostic factors for LCSS and OS in the training dataset
As shown in Table 2, independent risk factors for LCSS are age, sex, race, T descriptor, N descriptor, M descriptor, marital status, and treatment strategy. The above-mentioned variables are also independent prognostic factors for OS. In addition, although the site of the primary tumor is not a prognostic factor for LCSS (P=0.120), it is an independent risk factor for OS (P=0.048). Other variables, such as tumor grade and laterality, are not prognostic factors for either LCSS or OS.

Three-and five-year LCSS-predicting nomogram model
Based on the results of the Cox multivariate regression analysis, age, sex, race, marital status, T descriptor, N descriptor, M descriptor, and treatment strategies were selected to build the final nomogram model. The three-and five-year LCSS nomogram for the training dataset is shown in Figure 3. The treatment strategy shows the largest range of risk scores, followed by the T descriptor, indicating that these two factors have the greatest impact on the prognosis.

Validation of the nomogram model
The predicted patient survival probability curve Based on the established nomogram model, a risk score was calculated for each case in the training dataset, which ranged from 64 to 317 (median 207). Patients in the training dataset were further divided into four groups based on quartiles of calculated risk scores (group 1: <161, group 2: 161-207, group 3: 208-240 and group 4: >240). Kaplan-Meier survival curves were plotted for each risk level ( Figure 4). The survival time of different risk levels was statistically different (all P values <0.001), indicating that this nomogram model has a good ability to distinguish between high-risk and low-risk patients.

Calibration curve
Bootstrapping (1,000 repetitions) with resampling of the 1,669 patients was used to obtain a relatively unbiased estimate of the model's performance. Calibration plots for the prediction of three-and five-year LCSS are shown in Figure 5, demonstrating a high degree of agreement between the nomogram prediction using the model and actual observations. The C-index for LCSS prediction in the training dataset is 0.761 [95% confidence interval (CI), 0.754 to 0.768] with an AUC of 0.886.

External validation
An external validation using cases in the testing dataset was performed. The C-index and AUC for LCSS prediction in the testing dataset are 0.773 (95% CI, 0.765 to 0.781) and 0.876, respectively. A calibration curve shows good agreement between prediction and observation in the probability of three-and five-year LCSS ( Figure 6).

Comparison of predictive ability for LCSS in LCLC patients between the present nomogram model and the AJCC TNM staging system (8th edition)
The performance of the AJCC TNM staging system and the present nomogram model are shown in Table 3. The results show that the predictive ability of the current nomogram model is significantly better than the AJCC TNM staging system (P<0.001). The decision curve analyses show that the nomogram model has better net benefits compared with the AJCC 8 th edition TNM staging system (Figure 7).

Discussion
Currently, the prognosis of patients with LCLC or other types of NSCLC is predicted using the AJCC TNM staging system (17). It has been well established that LCLC has a higher degree of malignancy and a poorer prognosis than other cancers (2). Therefore, a prediction method based on the TNM stage is not accurate enough to meet the needs of LCLC patients. It is, therefore, necessary to establish a new tool for better predicting the prognosis of LCLC based on    routinely used clinicopathological variables. In this study, we have developed and validated a nomogram prognostic model using a large cohort of LCLC cases from the NIH/ NCI SEER database. This nomogram model, consisting of common demographics, staging, and treatment information, predicts the probability of long-term survival for individual LCLC patients very well. The C-index and AUC of the nomogram model in the training dataset were 0.761 (95% CI, 0.754 to 0.768) and 0.886, respectively. The performance of the model was similar in the testing set, with a C-index of 0.773 (95% CI, 0.765 to 0.781) and an AUC of 0.876, indicating that the nomogram model had a strong predictive ability. In addition, this model had better performance than the conventionally used 8 th edition AJCC TNM staging system, with a higher C-index (0.767 vs. 0.676, P<0.001) and an improved prediction of LCSS in LCLC patients. Thus, this comprehensive and personalized risk score calculation method might be used as stratification criteria and applied to clinical practice.
In 2019, He et al. (11) developed a nomogram model for LCNEC with a C-index of 0.75 for the training sets  (II) this study compared the impact of different treatment strategies on the prognosis of LCLC patients, which was in line with the actual situation of LCLC treatment and was easy for the clinician to use; (III) this nomogram was verified by various methods, and the results show that the nomogram was highly reliable.
Many clinical factors affect the survival of LCLC patients. The nomogram model developed in this study includes the following clinicopathological variables: age, sex, ethnicity, marital status, T descriptor, N descriptor, M descriptor, and treatment strategy. This nomogram model    (19) built a predictive survival model for LCSS and OS in NSCLC after radical resection and found that age >60 is associated with a worse OS. Gender also has been found to have an impact on the lung cancer patient prognosis over time (20,21). For example, Ferguson et al. (20) studied 772 patients with lung cancer and found that females outlived males (P<0.0001). Many previous studies have reported that the prognosis of NSCLC is affected by ethnicity. It was reported that Asian or Hispanic ethnicities have better survival rates compared to those with white or black ancestry (22,23). Our findings from the present study support this conclusion. Marital status also has an impact on survival in various types of cancer, such as breast, prostate, colorectal, gastric, and NSCLC (24)(25)(26). Wu et al. (26) carried out a population-based study of 70,006 patients with NSCLC and found that married patients have an advantage over the unmarried in both OS and LCSS. The explanation for this phenomenon is that married cancer patients are more likely to have better financial support, be diagnosed at earlier stages, receive recommended treatments, and make healthier choices due to support from their spouse (27). Treatment strategies significantly affected the prognosis of LCLC patients (28)(29)(30). Our study demonstrated that surgery was the most important treatment with the lowest risk score, and cases receiving no treatment had the highest risk score. Interestingly, we found that adjuvant therapy did not improve the prognosis of LCLC patients, compared with surgery alone, which is different from the treatment strategy of LCNEC (4). We analyzed this phenomenon and proposed the following reasons: LCLC is one of the pathological types of NSCLC and having similar biological characteristics as other NSCLCs. LCLC is frequently not sensitive to chemotherapy and surgical resection is the firstline treatment for operable LCLC (31). Regarding the effect of surgical treatment on LCLC, Hanagiri and his colleagues (32) conducted a survey of 975 patients who had undergone resection for NSCLC. He reported that the 5-year survival rate was 61.5% after surgery, and proposing that complete resection is the preferred treatment. However, sometimes, the prognosis for LCLC was dismal even after curative resection. In this case, adjuvant therapy was needed. The indication of adding adjuvant therapy should follow the treatment strategy of other NSCLCs.
It had been reported that platinum-based chemotherapy significantly improved OS (31). Although surgery yields the best prognosis for LCLC, the selection of treatment strategies should be made based on multiple factors, such as the TNM stage of the tumor, the patient's general conditions, and the ability of the hospital to provide a high level of comprehensive treatment for a patient. The optimal treatment plan for a patient should be determined only after the comprehensive evaluation. This study had several limitations. First, this was a retrospective study, and selection bias could therefore not be avoided. Second, it should be acknowledged that there are many variables affecting LCLC prognosis, including smoking and chronic obstructive pulmonary disease (COPD), which were not provided by the SEER database. However, our study had certain advantages as well. First, to our knowledge, our study is one of the only studies to predict the prognosis of LCLC patients in a nomogram model. Second, this study used a large-scale subject population to develop a nomogram and performed both internal and external validations, which made the nomogram model highly reliable. and five-year (B) lung cancer-specific survival. Horizontal line (Threshold probability) referred to the recurrence probability, which was predicted using the nomogram model or the 8 th edition TNM staging system. When the recurrence probability exceeded a certain value, the clinician should consider starting treatment for the patient. After starting the treatment, recurrence patients would be benefited from the treatment (benefit), and those who have not relapsed would be harmed by the treatment (disadvantage). The net benefit = benefit − disadvantage. Decision curve analysis was a tool to evaluate the net benefit of starting treatment. The grey lines represent extreme situations.
The horizontal line represents that all patients are untreated and the net benefit is zero. The oblique line indicates that all patients have been treated, and the net benefit is a backslash with a negative slope. Both the nomogram model (blue line) and the 8 th edition AJCC TNM staging system (red line) have obvious gains in net benefits compared to the extreme curve. The decision curve of the nomogram model is better than that of the AJCC TNM staging system.

Conclusions
In summary, we developed and validated a new nomogram model for predicting the prognosis of LCLC patients with good calibration. The nomogram performed well with satisfactory discrimination. The ability of this nomogram model to predict the prognosis was not inferior to the 8 th edition AJCC TNM staging system. This nomogram may be useful in assisting the clinician in predicting the oncological prognosis of LCLC patients and making decisions regarding appropriate treatment strategies.

Acknowledgments
We thank TopEdit (www.topeditsci.com) for its linguistic assistance during the preparation of this manuscript. Funding: None. Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the noncommercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.