{'fraction_non_alphanumeric': 0.037922305520397236, 'fraction_numerical': 0.02555739460617272, 'mean_word_length': 5.020808909730364, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Externally Validated Nomogram for Predicting Lymph Node Metastasis of Presumed Stage I and II Endometrial Cancer

Background: Optimal management for endometrial cancer in patients with clinically negative lymph nodes is still under debate. Several prediction models for lymphatic dissemination of early-stage endometrial cancer have been developed. However, external validation is rare, and decision curve analysis has hardly been applied for these models. Objective: To develop and validate a nomogram to predict lymph node metastasis of presumed stage I and II endometrial cancer. Study Design: The prediction nomogram was developed by using multivariable logistic regression with data for 700 EC patients who underwent initial surgery from 2006 to 2017 at Peking University People's Hospital (training dataset), Beijing. External validation was performed in 727 eligible patients from Fudan University Shanghai Cancer Center (validation dataset), Shanghai. Results: For the 700 women in the training dataset, the lymph node metastasis rate was 8.0% (56/700). Lymphovascular space invasion, histological grade, cervical stromal invasion, and myometrial invasion were independent prognostic factors in the training dataset. We generated a nomogram based on these pathological factors. To determine the clinical usefulness of our nomogram, we compared it with the Mayo criteria. For our nomogram, the area under the receiver operating characteristic curve (AUC) was 0.85 as compared with 0.63 for the Mayo criteria. In the validation dataset, the AUC was 0.78 as compared with 0.57 for the Mayo criteria. The nomogram was well-calibrated in both the training and validation datasets. At a 10% probability threshold, our nomogram decreased almost 29 unnecessary lymphadenectomies per 100 patients than the Mayo criteria without missing more metastatic disease. Conclusion: We developed a nomogram to predict lymph node metastasis in patients with early-stage endometrial cancer in China. This prediction model may help clinicians in decision-making for patients with early-stage endometrial cancer, especially for the patient with incomplete surgery, reducing overtreatment, and medical costs.


INTRODUCTION
Lymphadenectomy is controversial in early-stage endometrial cancer (EC). The net benefit for lymphatic dissemination in patients with early-stage endometrial cancer is difficult to estimate accurately. Furthermore, the contribution of lymphadenectomy to long-term survival of women with stage I and II endometrial cancer is ambiguous. Several studies have shown that lymphadenectomy was conducive to define the extent of disease lesions, guide adjuvant treatment, and improve prognosis (1)(2)(3). However, two large-scale randomized controlled trials hinted at opposite results. First, traditional lymphadenectomy could not improve disease-free survival or overall survival in EC patients (4,5); second, the incidence of complications, such as intestinal obstruction, lymphocyst, deep venous thrombosis, and other surgical complications, increases with expansion of the scope of surgery (6)(7)(8). Moreover, the risk of lymph node metastasis in early-stage endometrial cancer is only about 10% (9). Whether lymphadenectomy leads to overtreatment of most patients with early endometrial cancer and increases additional medical costs remains a question.
Currently, most clinicians in China follow the criteria for judging low-risk groups without lymphadenectomy proposed by the Mayo clinic in 2000. The Mayo criteria classify endometrial cancer patients at low risk with the variables: endometrioid endometrial cancer, tumor diameter <2 cm, grade 1 or 2, and myometrial invasion (MI) <50%. Vargas et al. analyzed 19,329 patients with endometrioid adenocarcinoma in the SEER database whose lesions were confined to the uterus, retrospectively (10). According to Mayo criteria, 78.9% of the patients were at high-risk for nodal metastasis, and the lymph node metastasis rate was 6.4%. Thus, almost 70% patients without lymph node metastasis were over-treated. Therefore, accurately predicting lymph node metastasis is crucial for selecting an appropriate surgical approach especially for earlystage endometrial cancer.
Recently, some practitioners have developed some prediction models for lymphatic dissemination by using nomogram, a precise calculation system based on a statistical algorithm (11)(12)(13)(14)(15). However, external validation was rare. The net benefit of decision curve analysis in lymph node dissection for earlystage endometrial cancer has not been calculated in the above studies. In this study, we developed and externally validated a parametric model of pathological characteristics based on a mathematical algorithm for predicting lymph node metastasis in early-stage endometrial cancer. Moreover, to determine the clinical usefulness of the nomogram, we used decision curve analysis and compared it with the Mayo criteria.

Study Population
From January 2006 to December 2017, the data of all patients who had received primary surgical treatment for endometrial cancer were abstracted from the database of Peking University People's Hospital. The medical records were reviewed to determine age, menopausal status, surgical procedure, tumor diameter, depth of myometrial invasion (MI), cervical stromal invasion status, histological type, histological grade, lymphovascular space invasion (LVSI) status, and final lymph node status. Tumor diameter was defined as the largest of the three dimensions of

Statistical Analysis
We estimated the sample size based on the area under the receiver operating characteristic curve (AUC). The AUC of our nomogram was estimated to be higher than 0.7. According to published articles, the risk of lymph node metastasis in earlystage endometrial cancer was about 10%, so we set the ratio between negative and positive cases as 9, with alpha error as 5% and test power as 95%. Therefore, our study should include at least 29 participants with lymph node metastasis and 261 participants without lymph node metastasis, which yielded a total sample size of 290 participants. Comparison of the training and validation cohorts involved chi-square or Fisher exact test for categorical variables. The nomogram was developed using univariate and multivariate logistic regression model. All parameters that were significantly associated with lymph node metastasis in univariate analysis were entered in the full model. Candidate variables with P values <0.05 were selected using a backward stepwise selection from the full multivariate model. The selected variables were used to develop the nomogram. The nomogram performance was assessed in both the training and validation groups by calculating discrimination and calibration. Discrimination is the ability to differentiate between women with lymph node metastasis and women without lymph node metastasis, and it is measured by AUC. Calibration is the agreement between the observed outcome frequencies and the predicted probabilities, and it is estimated by the Hosmer-Lemeshow goodness-of-fit test. The AUC ranges from 0 to 1, and a model is considered to have a poor, fair, or good performance if the AUC lies between 0.5 and 0.6, 0.6, and 0.7 or is >0.7, respectively. Calibration is reflected with average errors. The optimal cutoff point of the nomogram based on the training set was estimated by Youden's J index (16).
Decision curve analysis was used to quantify the clinical usefulness of the model. The clinical net benefit of the model: one sums the benefits (true positives) and subtracts the harms (false positives) at each threshold probability (Pt) of the outcome, and was calculated as: The net reduction presents the reduction of patients in unnecessary intervention per 100 patients, and was calculated as: Decision curve analysis of the nomograms and Mayo criteria involved using the Decision Curve package of R software.
All analyses were performed in SPSS v22.0 and R software version 3.4.4 (https://www.r-project.org/), using the rms, presence/absence, decision curve packages. P <0.05 was considered statistically significant.

Patients and Variables
The training cohort consisted of 700 eligible women with earlystage EC who underwent lymph node dissection (Figure 1). The external validation cohort included 727 patients. For the 700 women in the training cohort, the lymph node metastasis rate was 8.0% (56/700) vs. 11.8% (86/727) in the validation cohort ( Table 1). The cohorts were similar in age, menopausal status, histological type, and LVSI. However, more women in the validation than training cohort had tumor diameter >2, grade 3, deep myometrial invasion, and cervical stromal invasion.

Factors Associated With Lymph Node Metastasis
On univariate logistic regression analysis, deep myometrial invasion, cervical stromal invasion, histological type, histological grade, tumor diameter, LVSI were associated with lymph node metastasis in the training cohort. All of these variables were analyzed by multivariable logistic regression analysis using backward method. Independent risk factors associated with lymph node metastasis in the training cohort were LVSI, grade, cervical stromal invasion, and myometrial invasion ( Table 2). LVSI was a major predictor, with adjusted odds ratio (OR 6.5; 95% CI, 3.4-12.4; P < 0.001). Other predictors were depth of myometrial invasion ≥50% (OR 2.0; 95% CI, 1.1-3.9; P = 0.03), cervical stromal invasion (OR 3.5; 95% CI, 1.7-7.1; P < 0.001), grade 2 cancer (OR 2.9; 95% CI, 1.0-8.0; P = 0.046), and grade 3 cancer (OR, 3.6; 95% CI, 1.2-10.4; P = 0.021). The results of univariate and multivariable analysis in the validation cohort were in Table 3. LVSI, cervical stromal invasion and MI were also risk factors for lymph node metastasis in the validation cohort. But the effects of histological type and histological grade on lymph node metastasis slightly differed in the two cohorts.

Development and Validation of the Prediction Model
The nomogram constructed from the final multivariate model was presented in Figure 2. For a given patient, points were assigned to each of the predictor variables in the nomogram and a total score was derived from the sum of present variables.  The total score corresponds to a predicted probability of lymph node metastasis. The performance of the final model was assessed through discrimination and calibration. The AUC for the nomogram was 0.85 (95% CI, 0.80-0.90) as compared with 0.63 (95% CI, 0.60-0.67) for the Mayo criteria in the training cohort (P < 0.001; Figure 3A). In the validation cohort, the AUC was 0.78 (95% CI, 0.72-0.83), as compared with 0.57 (95% CI, 0.54-0.59) for the Mayo criteria (P < 0.001; Figure 3B). The Frontiers in Oncology | www.frontiersin.org  nomogram showed superior discrimination to the Mayo criteria both in the training and validation groups.
The predicted probabilities and the actual probabilities of lymph node metastasis in the training and validation groups are shown in the calibration plot (Figure 4). The nomogram was well-calibrated both in the training (mean absolute error = 0.012) and validation cohorts (mean absolute error = 0.013).

Optimal Threshold of the Model
Individualized scores for each patient were accurately calculated according the nomogram. Therefore, we used an optimal cut-off value that maximized the sum of sensitivity and specificity in the ROC curve. The best cut-off point was 80 points. According to the cut-off point, the patients, both in the training and validation cohorts, were divided into low-risk group (score<80 points) and  Figure 5). In the training cohort, the lymph node metastasis rates were 2.1 and 21.0% in low risk-group and highrisk group according to the nomogram, and those were 1.4 and 10.8% according to the Mayo criteria. In the validation cohort, the lymph node metastasis rates were 4.3 and 22% in low-risk group and high-risk group according to the nomogram, and those were 2.7 and 13.5% according to the Mayo criteria ( Tables 4, 5).

Decision Curve Analysis and Net Benefit
Comprehensive comparative analysis, combined with a net benefit curve, showed that with threshold probability 3%∼20%, the nomogram increased the net benefit by 0.2∼3.5 per 100 patients in the training cohort (Table 6, Figure 6A). We observed similar results in the validation cohort (Table 7, Figure 6B). At a 10% probability threshold, our nomogram led to 3.8 positive result per 100 patients without an increase in the number of falsepositive results, while Mayo can only lead to 0.6 positive patient. At this probability threshold, the nomogram also decreased 28.8 unnecessary interventions per 100 patients without missing more metastatic disease than the Mayo criteria.

DISCUSSION
Although lymph node status is a very important prognostic factor, the incidence of lymph node metastasis is only about 10% in clinically suspected early-stage endometrial cancer (9), and the published results of two randomized trials did not show survival benefit for routine lymphadenectomy in earlystage endometrial cancer (4,5). In addition, surgical morbidities increase by performing lymphadenectomy. Hence, systematic lymphadenectomy has become a matter of debate in earlystage endometrial cancer. Several risk classifications are used  to identify subsets of patients with lower risk of lymph node metastasis. The Mayo criteria was mostly used in China. However, according to the Mayo criteria, the lymph node metastasis rate for the high-risk group was 6.4%, more than 70% patients without lymph node metastasis were over-treated (10). The key to solve this dilemma is how to accurately identify potential lymph node metastasis. Sentinel lymph-node biopsy with less invasive intervention is being adopted in surgical treatment of EC (17). However, the significance of such biopsy in assessing lymph node metastasis has been found to be limited (18,19), and the prognostic value of the intervention is still under investigation. Based on this background information, we aimed to construct a prediction model that could identify patients who required systematic lymphadenectomy. Clinicopathological variables, such as histological type and grade, tumor diameter, depth of myometrial invasion, cervical stromal invasion, LVSI, CA-125 have been reported to be associated with lymph node metastasis. However, none of these parameters can accurately identify patients with lymph node metastasis individually. Several authors developed models to predict lymph node metastasis by combining multiple indicators (11-15) ( Table 8).
Aihilli et al. established a model for predicting lymph node metastasis in 883 women with early-stage endometrioid EC that included MI, grade, primary tumor diameter, cervical stromal invasion and LVSI (15). The internal validation of the nomogram showed a good discrimination (AUC = 0.88), but the author did not validate the model to assess the applicability and to an independent population. Bendifallah et al. performed an external validation of the prediction model in 322 patients with earlystage endometrioid EC (20). They found the model had a fair discrimination with a C-index of 0.65. Then, Bendifallah et al. developed a nomogram with data for 523 early-stage EC women based on four factors including grade, LVSI, MI, and tumor diameter, with a C-index of 0.83 (13), but there was no external validation either.
There are three other nomograms to predict lymph node metastasis in endometrial cancer. Kang et al. analyzed the clinical data for 307 cases of stage I-IV endometrial cancer with 31 cases (11.1%) had lymph node metastasis. On multivariate regression analysis, LVSI, grade, MI and tumor diameter were selected as high-risk factors of lymph node metastasis. The C-index of the model was 0.87, which had very good discrimination (14), but the number of cases was limited and external validation was not performed. Pollom et al. developed a prediction   On multivariate regression analysis, CA125, histological type, cervical interstitial involvement and LVSI were selected as high-risk factors of lymph node metastasis. The C-index of the model was 0.92, which had a very good discrimination (11), but the number of cases was limited and external validation was not performed either. Owing to small sample size, these three nomograms included early and advanced endometrial cancer and no external validation was performed. In our study, LVSI, MI, grade, and cervical stromal invasion were selected to construct the nomogram using multivariable logistic regression analysis. These four high-risk factors predicting lymph node metastasis were also present in previous studies. Seven hundred and twenty-seven patients with endometrial cancer, from Fudan University Shanghai Cancer Center between 2006 and 2016, were used to validate the nomogram. Our data showed that the discriminations were good both in the training cohort (AUC = 0.85) and in the validation cohort (AUC = 0.78). As the Mayo criteria was mostly used in China, we compare our nomogram with it. Our nomogram was better than the Mayo criteria. The clinical decision-making curve shows that, within a threshold probability ≥3.0%, patients will receive more net benefit from application of the nomogram than Mayo criteria. We use 80 points as the cut-off point based on Youden's J index. The patients were divided into low-risk group (score<80 points) and high-risk group (score>80 points). The performance comparison of nomogram stratification and Mayo criteria for predicting lymph node metastatic was verified. The nomogram showed a better discrimination than the Mayo criteria in both training (with AUC of 0.78, 95% CI, 0.72-0.83 vs. 0.63, 95% CI, 0.60-0.67; P < 0.001) and validation cohorts (with AUC of 0.71, 95% CI, 0.66-0.75, vs. 0.57, 95% CI, 0.54-0.59; P < 0.001). Thirty-two percent of the patients were classified as high-risk group according to the nomogram stratification, compared with 70 percent based on the Mayo criteria. The lymph node metastasis rates were 2.1 and 21.0% in low riskgroup and high-risk group, according to the nomogram, while those were 1.4 and 10.8% according to the Mayo criteria. At a 10% probability threshold, our nomogram led to 3.8 positive result per 100 patients without an increase in the number of false-positive results, while Mayo only can lead to 0.6 positive patient. On the other hand, at this probability threshold, the nomogram can decrease 28.8 unnecessary interventions per 100 patients without missing more metastatic disease than the Mayo criteria.
There are some limitations in the current research. First, our model was established by applying postoperative indicators, so it is mainly used to evaluate whether lymphadenectomy should be performed after incomplete surgery. Second, our data were not subdivided because of the limited number of cases. Lymph node metastasis was not divided into pelvic and para-aortic lymph node metastasis in the cohorts. Type I and II were not considered separately. In addition, the research was retrospective study, and prospective external validation was needed to verify the promotion and application of the model.

CONCLUSION
In conclusion, our research involved the development and external validation of a feasible prediction model of lymph node metastasis in early-stage endometrial cancer. The nomogram to predict lymph node metastasis had better accuracy and net benefit than the Mayo criteria. Calibrations were relatively good between the two institutions. The newly developed nomogram can preferably guide decision-making for surgery for women with early-stage EC to avoid overtreatment and unnecessary costs.

DATA AVAILABILITY STATEMENT
The datasets generated for this study are available on request to the corresponding author.

ETHICS STATEMENT
Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

AUTHOR CONTRIBUTIONS
JW and HW mainly designed this project and revised the manuscript. YD and YC were responsible for collecting data from Peking University People's Hospital, sorting up and statistical data, writing and revising the manuscript. HZ mainly gave guidances in data statistics and R language programming. WT, BS, and YR mainly established the database of endometrial cancer form Fudan University Shanghai Cancer Center. ZW, XL, and LW gave the suggestions in clinical value and application of the predictive model.