{'fraction_non_alphanumeric': 0.041346924750887644, 'fraction_numerical': 0.03212690413469248, 'mean_word_length': 4.865972455492106, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Comparison Between the Online Prediction Models CancerMath and PREDICT as Prognostic Tools in Thai Breast Cancer Patients

Background and Purpose Web-based prognostic calculators have been developed to inform about the use of adjuvant systemic treatments in breast cancer. CancerMath and PREDICT are two examples of web-based prognostic tools that predict patient survival up to 15 years after an initial diagnosis of breast cancer. The aim of this study is to validate the use of CancerMath and PREDICT as prognostic tools in Thai breast cancer patients. Patients and Methods A total of 615 patients who underwent surgical treatment for stage I to III breast cancer from 2003 to 2011 at the Division of Head Neck and Breast Surgery, Department of Surgery, Siriraj Hospital, Mahidol University, Thailand were recruited. A model-predicted overall survival rate (OS) and the actual OS of the patients were compared. The efficacy of the model was evaluated using receiver-operating characteristic (ROC) analysis. Results For CancerMath, the predicted 5-year OS was 88.9% and the predicted 10-year OS was 78.3% (p<0.001). For PREDICT, the predicted 5-year OS was 83.1% and the predicted 10-year OS was 72.0% (p<0.001). The actual observed 5-year OS was 90.8% and the observed 10-year OS was 82.6% (p<0.001). CancerMath demonstrated better predictive performance than PREDICT in all subgroups for both 5- and 10-year OS. In addition, there was a marked difference between CancerMath and observed survival rates in patients who were older as well as patients who were stage N3. The area under the ROC curve for 5-year OS in CancerMath and 10-year OS was 0.74 (95% CI; 0.65–0.82) and 0.75 (95% CI; 0.68–0.82). In the PREDICT group, the area under the ROC curve for 5-year OS was 0.78 (95% CI; 0.71–0.85) and for 10-year OS, it was 0.78 (95% CI; 0.71–0.84). Conclusion CancerMath and PREDICT models both underestimated the OS in Thai breast cancer patients. Thus, a novel prognostic model for Thai breast cancer patients is required.


Introduction
Breast cancer is a heterogenous disease in terms of various biological and clinical behaviors. 1,2 The prognosis of early breast cancer has significantly improved with adjuvant treatments. The combinations of optimal chemoendocrine regimens have resulted in 57% and 45% reductions in 15-year mortality for women under the age of 50 and those aged 50-69, respectively. 3 However, among breast cancer patients who meet the eligibility criteria for adjuvant systemic therapy, it has been argued that up to 60% of these patients experienced loss of quality of life due to toxicity with little or no survival benefit. 4 The combined impact on quality of life as well as a rise in financial costs mean the avoidance of unnecessary treatment is a crucial goal for patients, oncologists, and policymakers. As a result, many models relating to risk prediction have been developed in order to aid oncologists in selecting patients who would benefit overall from systemic therapy.
At present, a number of prognostic tools such as multigene assays or free web-based prognostic calculators are available to aid decision-making in administering adjuvant systemic therapy. In more economically-developed nations, commercial gene expression-based prognostic tests such as Oncotype Dx and Mammaprint are widely implemented but these tools are inaccessible for many countries due to their expensive costs. 5 Therefore, online prognostic calculators are valuable in clinical practice. There are several free web-based prognostic calculators to predict the survival of early breast cancer such as Adjuvant! Online, 6 CancerMath, 7 and PREDICT. 8 These have been validated in Western populations and deemed to be calibrated and accurate in discrimination. [9][10][11] CancerMath was proposed in 2009 (http://www.life math.net/cancer/breastcancer/outcome/index.php). It predicts the risk of mortality for the first 15 years after diagnosis, any reduction in life expectancy, and any impact of adjuvant treatments. PREDICT, another prognostic model, was established in 2010 (https://breast.predict.nhs. uk). The newest version of PREDICT provides predicted 5-, 10-, and 15-year overall survival (OS) with additional benefits of each adjuvant therapy. There have been a small number of studies to confirm the prognostic ability of these programs in Asian populations. 12,13 Despite widespread use of both CancerMath and PREDICT, data comparing the validity between these two models are limited and controversial. Lass E and colleagues demonstrated that CancerMath and PREDICT had similar outcomes in American and Canadian breast cancer patients who were estrogen receptor (ER)-positive and human epidermal growth factor receptor 2 (HER2)-negative. 14 On the other hand, a study comparing the two models in the United Kingdom illustrated that PREDICT outperformed CancerMath in terms of discriminatory accuracy, calibration, and clinical utility. 15 Genetic backgrounds, socio-economic status, and cultures in Asia are all different factors that play distinct roles in the prognosis and treatment of breast cancer among Asians and people in America and Europe. Thus, the aim of this study was to validate the CancerMath and PREDICT models in Thai breast cancer patients. Breast cancer patients who underwent surgical treatment for  stage I to III breast cancer, according to the American Joint  Committee on Cancer Staging Manual seventh edition, from  2003 to 2011 at the Division of Head Neck and Breast Surgery, Department of Surgery, Siriraj Hospital, Mahidol University, Thailand were recruited. No informed consent was required because this study collected existing data in a manner that the subjects could not be identified. This study was approved by the Siriraj Institutional Review Board (certificate of approval number Si 732/2018) and conducted in accordance with the Declaration of Helsinki. The inclusion criteria of patients were age between 25 and 85 years old with unilateral breast cancer with tumor size T1-T3, available axillary nodal status, no distant metastasis, available ER/progesterone receptor (PR)/HER2/and Ki-67 status, and those who had received complete surgical treatment (Mastectomy or breast-conserving surgery followed by radiotherapy). Exclusion criteria were those who were receiving neoadjuvant therapy whose tumor size, tumor grade, or adjuvant treatment status were unknown. Patients who were lost to follow-up after surgery were also excluded. Patient data was reviewed from their medical records. The ER and PR status were defined as positive when there was 1% to 100% of tumor nuclei positive. 16 HER2 immunohistochemistry was interpreted as follows: score 0 and 1+ =negative; score 2+ =equivocal; and score 3+ =positive. HER2 equivocal by immunohistochemistry was further determined by dual in situ hybridization (DISH). The ratio of HER2/CEP17 of ≥2 was defined as positive HER2. 17 Follow-up time was calculated from the date of surgery to the date of death or date of last follow-up. The date of last data collection was December 1st, 2018. The final dataset for validation of 5-year and 10-year survival was composed of 545 and 397 patients, respectively.

Predicted 5-and 10-Year OS by PREDICT
For the PREDICT model version 2.1, the data used to calculate predicted OS included age at diagnosis (between 25 and 85), postmenopausal status (yes, no, or unknown), ER status (positive or negative), HER2 status (positive, negative, or unknown), Ki-67 status (>10%, ≤10%, or unknown), invasive tumor size (mm), tumor grade (1, 2, or 3), detection method (screening, symptom, or unknown), number of positive nodes, hormone therapy (no or yes), chemotherapy (none, second generation, or third generation), trastuzumab (no or yes), and bisphosphonates (no or yes). Categorization of chemotherapy generation was different from CancerMath. There was no first-generation regimen included. Anthracycline-based regimens such as FEC were defined as second-generation chemotherapy and chemotherapy regimens that contained taxanes were categorized as third-generation chemotherapy. Because of this difference, first-generation regimens were registered as second-generation regimens. The calculator predicted 5-, 10-, and 15-year OS for 5 scenarios as follows; without adjuvant treatment, additional benefit of adjuvant hormone therapy, additional benefit of adjuvant chemotherapy, additional benefit of trastuzumab, and additional benefit of bisphosphonates. Predicted OS for 5 and 10 years with additional benefit of adjuvant treatment were collected.

Statistical Analysis
Kaplan-Meier analysis was used to estimate the observed 5 and 10-year OS in the entire study population and within subgroups. The CancerMath and PREDICT 5-and 10-year OS were calculated by individual predicted survival probabilities. The t-test and p-values of <0.05 from the comparison between model-predicted OS and patients' actual OS represented a significant difference between predicted and observed survival. Moreover, the multivariable regression coefficients were applied to estimate the relation between parameters. The difference between actual and predicted survival and the ability of the models was analyzed using receiver-operating characteristic (ROC) analysis. The area under the ROC curve (AUC) demonstrated the discriminatory performance of the model. An AUC of 0.5 demonstrated no discriminative performance and an AUC of 1.0 illustrated perfect discrimination. Analyses were performed using SPSS for Windows version 22.0 (SPSS Inc., Chicago, IL, USA).

Comparison of Observed and Model Predicted 5-and 10-Year OS
Overall, CancerMath and PREDICT models underestimated both 5-and 10-year OS. The predicted 5-year OS by CancerMath and PREDICT were 88.9% and 83.1%, respectively, whereas the observed 5-year OS was 90.8% (p<0.001). The predicted 10-year OS via CancerMath and PREDICT were 78.3% and 72.0%, respectively, whereas the observed 10-year OS was 82.6% (p<0.001). Additionally, CancerMath had a higher accuracy in prediction than PREDICT in all subgroups for both 5-and 10-year OS, especially for breast cancer patients with T3 tumors, metastases in 4 or more lymph nodes, or who were classified as stage III (Table 2).
CancerMath demonstrated a more noticeable underprediction of OS in patients older than 60 years (mean difference −6.3% for 5-year OS and −2.1% for 10-year OS), tumor size smaller than 2 cm (mean difference −4.3% for 5-year OS and −7.2% for 10-year OS), and ER or PR positive with HER2negative subtype (mean difference −4.4% for 5-year OS and −6.6% for 10-year OS). Furthermore, the underestimation of OS also occurred in patients who received adjuvant treatment with third-generation chemotherapy (mean difference −11.3% for 5-year OS and −24.4% for 10-year OS) or hormonal therapy containing AI (mean difference −6.6% for 5-year OS and −11.1% for 10-year OS).
In contrast, CancerMath significantly overpredicted survival in patients with N3 (≥10 positive nodes) (mean difference 15.1% for 5-year OS and 11.8% for 10-year OS) and patients receiving ovarian ablation plus tamoxifen (mean difference 8.3% for 5-year OS and 7.2% for 10-year OS). The model seemed to overestimate short-term OS in triplenegative breast cancer, even though it was not appeared in predicted long-term OS as shown in Tables 2 and 3.
According to multiple regression analysis, the factors that revealed significantly large standardized coefficients were age >60 year (0.39 at 5-year OS and 0.43 at 10-year OS), N3 status (0.21 at 5-and 10-year OS), and stage III (0.28 at 5-year OS and 0.35 at 10-year OS).         Figure 1).

Discussion
Accurate prognostic tools are crucial for individualized treatment of breast cancer. This study validated two models (CancerMath and PREDICT) in Thai breast cancer patients and the results showed that both models underpredicted 5-and 10-year OS in the overall dataset. Between the two, CancerMath predicted outcomes more accurately than PREDICT in all subgroups. Both CancerMath and PREDICT performed fair discrimination between survivors and nonsurvivors at 5-and 10-years after diagnosis. To our knowledge, this is the first study to validate the outcomes between CancerMath and PREDICT in Thai breast cancer patients. In Asian populations, a previous study by Wong et al demonstrated the validity of using PREDICT in most subgroups of patients recruited from University Malaya Medical Centre. The model-predicted 5-and 10-year OS of 86.3% and 77.5%, respectively, whereas the actual 5-and 10year OS were 87.6% and 74.2%. 12 For the PREDICT model, our results and the previous study were discordant even though the calculated OS was similar. The better observed OS in our study may be due to the inclusion of fewer patients with poor prognostic factors such as locally advanced stage and HER2 positive breast cancer.
There are a number of factors that may have resulted in the PREDICT model underestimating survival in this study. First, only the ER status was collected in PREDICT and therefore PREDICT could not account for patients with negative ER but positive PR (n=37, 6.0%). Second, PREDICT excluded first-generation chemotherapy which was used in 46 patients (7.5%), thus potentially resulted in an inaccurately calculated OS. Finally, 602 patients (97.9%) had no Ki-67 status which also contributed to inaccurate calculation of predicted survival.
The study by Miao et al that validated CancerMath in the Singapore Malaysia Hospital-Based Breast Cancer Registry revealed that the model slightly overestimated survival. The CancerMath predicted and observed OS probabilities were 87.3% and 83.4% at 5 years while they were 75.3% and 70.4% at 10 years after diagnosis. In addition, an overestimated OS in patients with N3 (≥10 positive nodes) was also found which corroborates the results of the current study. In the study by Miao, CancerMath also showed fair discrimination with an AUC for 5-year and 10-year OS of 0.77 (95% CI; 0.75-0.79) and 0.74 (95% CI; 0.71-0.76) respectively. 13 In this study, the greatest overpredicted OS of CancerMath was found in the subgroup of patients with N3. In this subgroup, 37.2% (n=19) were HER2 positive breast cancer but only 3.9% (n=2) were treated with trastuzumab as adjuvant treatment. Among 31.4% (n=16) of postmenopausal patients with positive ER or PR status, 7 patients (13.7%) received tamoxifen instead of AI. Furthermore, 84.3% (n=43) of this subgroup received radiation therapy. Owing to undertreatment, the actual survival of this subgroup was lower than those calculated by CancerMath. Additionally, CancerMath had only been validated among patients with a number of positive nodes of less than 10 while 8.3% (n=51) of the patients in our dataset had more than 10 positive nodes. Finally, many studies claimed that Asian breast cancer patients tend to manifest with characteristics consistent with poorer prognosis than their Western counterparts, such as more advanced stage, younger age, and more aggressive subtypes including HER2 overexpression and basal-like subtype; 18,19 thus, the predicted OS from models based on Western populations might overestimate OS in Asian patients.
Women with added ovarian ablation to tamoxifen was another group that had overestimated OS using CancerMath. This phenomenon may have been caused by two conditions. First, in this cohort, 151 premenopausal patients with hormonal receptor positive breast cancer who had a risk of recurrence were supposed to get tamoxifen plus ovarian ablation to permit adjuvant chemotherapy. However, none of these patients ended up receiving ovarian ablation. Second, there were a small cohort of patients in ovarian ablation plus tamoxifen subgroup (n=18). This implies that a greater emphasis on intensive treatment policies for locally advanced breast cancer and premenopausal women with positive ER or PR must be considered in Thailand.
The most underestimated OS using CancerMath occurred in patients who received third-generation chemotherapy. This may be explained by the small patient cohort (n=12) and the absence of any mortalities amongst this subgroup. CancerMath also underestimated OS in patients using AIcontaining regimens, which happens to be the group of patients aged >60 years who are assumed to be the group to receive AI. The authors hypothesized that the high rate of discontinuation of AI in Western populations (32.4%-36.2%) 20,21 and the low rate of full adherence (49%) in patients may have contributed to this outcome. 22 A few studies based on Asian populations revealed that the rate of interruption and non-adherence to AI was 14.9% and 15.4%-22.7% in adjuvant hormonal therapy, respectively. 23,24 Several limitations of this study should be considered. This study was a retrospective study and hence, there may be selection bias. This study was also conducted in a single center with a small number of patients; thus, a multicenter study looking at using these models in Thai populations may address this limitation.

Conclusion
Both CancerMath and PREDICT models underestimated OS in Thai breast cancer patients and CancerMath outperformed PREDICT in predictive performance in all subgroups for both 5-and 10-year OS. The clinical application of CancerMath should not be implemented to patients with N3 and aged over 60. Therefore, a novel prognostic model for Thai breast cancer patients is urgently required.