{'fraction_non_alphanumeric': 0.03277103255614054, 'fraction_numerical': 0.018212098420543676, 'mean_word_length': 5.290976681311254, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 2, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Impact of deep learning-determined smoking status on mortality of cancer patients: never too late to quit

Background Persistent smoking after cancer diagnosis is associated with increased overall mortality (OM) and cancer mortality (CM). According to the 2020 Surgeon General's report, smoking cessation may reduce CM but supporting evidence is not wide. Use of deep learning-based modeling that enables universal natural language processing of medical narratives to acquire population-based real-life smoking data may help overcome the challenge. We assessed the effect of smoking status and within-1-year smoking cessation on CM by an in-house adapted freely available language processing algorithm. Materials and methods This cross-sectional real-world study included 29 823 patients diagnosed with cancer in 2009-2018 in Southwest Finland. The medical narrative, International Classification of Diseases-10th edition codes, histology, cancer treatment records, and death certificates were combined. Over 162 000 sentences describing tobacco smoking behavior were analyzed with ULMFiT and BERT algorithms. Results The language model classified the smoking status of 23 031 patients. Recent quitters had reduced CM [hazard ratio (HR) 0.80 (0.74-0.87)] and OM [HR 0.78 (0.72-0.84)] compared to persistent smokers. Compared to never smokers, persistent smokers had increased CM in head and neck, gastro-esophageal, pancreatic, lung, prostate, and breast cancer and Hodgkin's lymphoma, irrespective of age, comorbidities, performance status, or presence of metastatic disease. Increased CM was also observed in smokers with colorectal cancer, men with melanoma or bladder cancer, and lymphoid and myeloid leukemia, but no longer independently of the abovementioned covariates. Specificity and sensitivity were 96%/96%, 98%/68%, and 88%/99% for never, former, and current smokers, respectively, being essentially the same with both models. Conclusions Deep learning can be used to classify large amounts of smoking data from the medical narrative with good accuracy. The results highlight the detrimental effects of persistent smoking in oncologic patients and emphasize that smoking cessation should always be an essential element of patient counseling.


INTRODUCTION
While smoking is a well-established risk factor for the development of many types of cancer, the 2020 report from the Surgeon General of the US Department of Health and Human Services emphasizes the need to assess the health effects of smoking cessation. 1 It has been estimated that two-thirds of patients who smoke continue to do so after cancer diagnosis. 2,3 This is alarming, since there is accumulating evidence that persistent smoking after cancer diagnosis impairs cancer-specific survival in multiple cancer types, 4 including small-cell and non-small-cell lung cancer (SCLC and NSCLC, respectively), 5,6 prostate, 7,8 head and neck, 9,10 colorectal, 11,12 and bladder cancer. 13 In breast cancer, conflicting results are reported, but large metaanalyses have demonstrated that persistent smoking is associated with poorer breast cancer-specific survival, especially in heavy smokers. 14,15 Deep learning is an advanced subtype of artificial intelligence-based machine learning, which is making its way into clinical medicine via various diagnostic and predictive applications. 16,17 This includes universal language modeling of medical narratives, wherein real-life practice health data, such as smoking status, are often presented in an unstructured format. 18 First reports of using language modeling to define an individual's smoking status appeared encouraging both from electronic health records (EHRs) 18,19 and from usergenerated content on a smoking cessation support website. 20 With the exception of lung and head and neck cancer, the effect of smoking cessation at the time of cancer diagnosis on cancer mortality (CM) remains inconclusive. 4,5,21,22 The aim of the current study was to evaluate the impact of smoking in cancer patients using deep learning, which allows us to efficiently study large patient populations. We applied this experimental setting on cancer patients diagnosed in the Turku University Hospital region, and subsequently analyzed (i) the effect of the individual's smoking status and (ii) the effect of recent smoking cessation on survival after cancer diagnosis.

Ethics
The study was approved by the Institutional Review Board of Turku University Hospital, Turku, Finland (T132/2019). No informed consent was required according to the Finnish Act on Secondary Use of Health and Social Data. Based on the act, EHRs were reviewed and data were stored in a secured analysis environment protected by the hospital firewall.

Study population
Turku University Hospital is a tertiary referral Organization of European Cancer Institutes-designated cancer center covering the entire region with 480 000 inhabitants. The neighboring regions of Satakunta and Vaasa, with populations of 220 000 and 66 000, respectively, may refer patients to Turku for cancer care. Since 2004, Turku University Hospital uses a real-time updated EHR system. The EHRs were searched for all patients with at least one International Classification of Diseases-10th edition (ICD-10) C code, excluding non-melanoma skin cancer (C44). All patients whose first ICD-10 C code was recorded during 2009-2018 were included in the current study ( Figure 1).

Definition of source data
The unique Finnish personal identification number was used to link patient-level data from different information systems, including ICD-10 codes, histology [International Classification of Diseases for Oncology-third edition (ICD-O-3) codes], Nordic operational codes [Nordic Medico-Statistical Committee (NOMESCO)], and chemotherapy [Anatomical Therapeutic Chemical (ATC) Classification codes] and radiotherapy records, to the medical narrative. For each patient, their medical narrative from 2004 to 2019 including on average 5600 words was extracted for language modeling. An extensive cross-linking of structured data was carried out to cross-validate all cancer diagnoses as accurately as possible. In case of discrepancy, such as a patient presenting with multiple ICD-10 codes, histology was used to establish the dominant diagnosis.
Date and cause of death were verified from Statistics Finland, an independent national registry. This verification also excluded non-Finnish citizens and ascertained complete follow-up. The number of incident cancer cases per region was obtained from the online statistics of the Cancer Registry of Finland (https://cancerregistry.fi/statistics/cancerstatistics/) to assess the population coverage.
EHRs were searched for body mass index (BMI), presence of select comorbidities, and Eastern Cooperative Oncology Group (ECOG) performance status. We defined synchronously metastatic cancer as presence of a metastasis code (C77-79) within 6 months of primary tumor diagnosis.

Language modeling for smoking status
For the current study, we built and used two language models, ULMFiT and Google BERT, 23,24 both of which use transfer learning. Firstly, a large amount of unlabeled data is leveraged to pre-train a model that performs well in the task of guessing masked words in phrases from the training data. The model learns a useful mathematical representation for the words and their connections. In this pre-training phase, the learned knowledge is transferred to a classifier supplemented with smaller amounts of manually labeled training data. The strength of this approach is that these models can be pre-trained with freely available Finnish text, and then manually fine-tuned with smaller clinical datasets, which in medicine are often hard and expensive to obtain.
Since no pre-trained models for Finnish were publicly available when we initiated the study, we pre-trained the ULMFiT model with the entire Finnish Wikipedia 2019 using the computer infrastructure at our hospital. For BERT, a pretrained model for Finnish was later available. 24 The learned knowledge was then transferred to a training classifier, by randomly picking 5000 tobacco smoking-related sample phrases and sentences from the medical narrative archive of our hospital, using the Finnish word-stem 'tupak' equivalent to the English word-stem 'smok'. 19 These sample phrases were manually labeled into three classes (never, former, or current smoker). ULMFiT-and BERT-based classification models were then trained on this data to produce smoking phrase classifiers.
To define an individual's tobacco smoking status, a logic was added to both models when multiple classifications were present over time. Individuals labeled as both never and former smokers were classified as former smokers. In any other case, the average probability for each class over the patient's sentences was calculated for each smoking status classification, and the highest probability was used to determine the final status.
After language model classification, recent quitters were identified by extracting cessation year, where available, for ULMFiT-defined former smokers. Recent quitters were those who quit within 1 year before or any time after cancer diagnosis. 1

Statistical analysis
The primary outcome measure was CM, defined as death due to any cancer. The secondary endpoint was overall mortality

ESMO Open
A. Karlsson et al.
(OM), defined as death due to any cause. All patients were censored at the time of death or at the end of follow-up in December 2019. Firstly, hazard ratios (HRs) were calculated with the Cox proportional hazards model with 95% confidence interval (CI 95 ), using sex as the categorical covariate with the enter method to control confounding. In case of significant sex interaction (P < 0.05), men and women were subsequently analyzed separately. Secondly, multivariate Cox regression was carried out, adjusting HRs for age >70 years, presence of synchronous metastasis (solid tumors only), ECOG 2 or more, presence of select comorbidities (Supplementary Table S1, available at https://doi.org/10.1016/j.esmoop.2021.100175), and BMI <20 kg/m 2 . SPSS version 26 (IBM, Armonk, NY) was used. Sensitivity and specificity analyses were calculated with 2 Â 2 contingency tables separately for never, former, and persistent smokers, excluding patients with missing smoking status, 19 using the Python 'sklearn' package.

RESULTS
The cohort consisted of 29 823 patients (14 717 females (Figure 2). Similar results, but smaller in size, were observed in female   Figure 2]. Results with OM appeared similar but were of lesser magnitude ( Figure 2). Specifically, persistent smokers had increased CM in head and neck cancer, gastro-esophageal cancer, colorectal cancer, NSCLC, SCLC, cutaneous melanoma (men only), prostate cancer, bladder cancer (men only), Hodgkin's lymphoma, and both lymphoid and myeloid leukemia (Figure 3). In former smokers, a trend favoring higher CM was observed in the abovementioned cancers, being significant in NSCLC, SCLC, bladder and prostate cancer, and lymphoid leukemia (Supplementary Figure S1,   and neck, gastro-esophageal, pancreatic, NSCLC, SCLC, breast, and prostate cancer and in Hodgkin's lymphoma. Thus, smoking emerged as a significant covariate in pancreatic and breast cancer, while significance was no longer observed in cutaneous melanoma and colorectal cancer (Table 1).

Recent quitting reduces cancer mortality
Smoking cessation year could be extracted for 2803 (48%) former smokers. An additional 41 recent quitters were identified during the manual validation process; thus, a total of 524 patients met the criteria for recent quitter. 1 Compared to those who continue smoking, recent quitting reduced both CM [HR 0.80 (0.74-0.87),

DISCUSSION
The recent Surgeon General's report emphasized the need to assess the health effects of smoking cessation at the time of cancer diagnosis. 1 Motivated by this report, we applied deep learning-based language modeling to analyze the medical narratives of a large cross-sectional cohort of cancer patients. With good population-based coverage, we confirmed the detrimental effect of persistent smoking on CM and OM in many types of cancer (head and neck, colorectal, NSCLC, SCLC, melanoma, breast, prostate, bladder, and myeloid leukemia). Interestingly, persistent smoking increased CM in gastro-esophageal and pancreatic Cox multivariate analysis includes presence of synchronous metastasis (solid tumors only), age >70 years, sex, ECOG 2 or more, body mass index <20 kg/m 2 , and presence of select comorbidities (diabetes, coronary heart disease, cerebrovascular disease, chronic obstructive pulmonary disease, and liver failure). Significant results are given in bold. CI 95 , 95% confidence interval; HR, hazard ratio; NSCLC, non-small-cell lung cancer; SCLC, small-cell lung cancer. cancer, lymphoid leukemia, and Hodgkin's lymphoma, to our knowledge not previously extensively reported in European populations. Furthermore, a major finding was that patients who quit smoking not earlier than 1 year before diagnosis showed significantly improved cancer and overall survival, including those with synchronously metastatic disease. One major strength of the current study is that we were able to link extensive structured clinical data to smoking data, and showed that smoking was an independent modifiable risk factor for CM in multiple cancer types. While the detrimental effects of persistent smoking have been shown before, 4,21 our results suggest that smoking cessation is beneficial irrespective of patient's age, comorbidities, or metastatic disease. We also analyzed OM along with CM and observed that OM HRs rarely exceeded CM HRs, suggesting that cancer patients who continue smoking will probably die due to cancer and not from other tobaccorelated diseases.
Artificial intelligence and deep learning applications are making their way into clinical practice, but their efficacy is not yet proven in prospective trial settings. 16 The current study utilized more advanced deep learning compared to previous reports [18][19][20] in a retrospective proof-of-principle setting, where we successfully extracted a large amount of smoking data in a matter of days. Two language models were tested with good specificity (88%-98%), comparable to the previous results in English language. 19 The BERT model included a pre-trained model for Finnish, 24 but with seemingly only slightly improved specificity in persistent smokers compared to ULMFiT. Discrepancies were most commonly due to unsuccessful attempts to quit, and the individual's smoking status was ultimately based on probability logic instead of on the language model itself. Misclassifications also had human reasons behind them; elderly people who tried smoking in their youth are former smokers by Definition, while human classification may overlook sporadic smoking. Furthermore, the amount of missing data was almost always true missing data, which may reflect healthcare professionals' current practices and attitudes toward reporting smoking behavior of their patients. 25 In the current study, 524 (8%) smokers at the time of cancer diagnosis managed to quit smoking and remain abstinent. This 8% is probably an underestimate, since the cessation year was found only in 48% of former smokers, but may reflect the hardships of smoking cessation in a realworld population. For comparison, abstinence rates of 36%-43% have been reported in large surveys and smoking cessation programs. 2,3,22 Regardless of missing cessation years, our study showed a reduced CM and OM in recent quitters compared to persistent smokers. We observed that former smokers were more often ECOG 0-1 compared to smokers, without explanatory sex or age differences.
One novel observation in our study is that smoking was an independent risk factor for Hodgkin's lymphoma mortality. However, the current study was not specifically designed to address predictive factors in lymphomas, and survival may be affected by a variety of unavailable covariates, including staging of hematological malignancies. The same holds also true for lymphoid leukemia, while in myeloid leukemia the detrimental effect of smoking has been reported earlier. 26 While smoking is a well-known adverse factor in esophageal cancer, 1 in pancreatic and gastric cancer smoking data are scarce or derived from Asian populations. 27,28 Results on cutaneous melanoma should be observed with caution, since detrimental effects of smoking were observed in men only and not significant once adjusted for other covariates, but are in line with previous reports. 4,21 The limitations of our study include its retrospective single-center nature and that we used only Finnish texts. However, ULMFiT and BERT are designed as universal language models, and the same methodology can be applied in any language provided enough data are available. 19,23,24 We acknowledge that our results should be reproduced in different languages before wider implementation. A possible limitation of our study design is that language models such as ours are trained on ordinary, everyday language from Wikipedia pages and discussion forums. Medical text is typically laden with technical terms that do not appear in everyday discussions, which could inhibit the transferability of what the model has learned in the pretraining phase.

Conclusions
To our knowledge, this is the largest comprehensive deep learning-based registry of smoking behaviors in cancer patients. We applied this experimental setting to link the individual's smoking status to cancer outcome and demonstrated the detrimental effects of persistent smoking on CM across multiple types of cancer independent of age, presence of metastatic disease, BMI, performance status, or comorbidities. The current study suggests that cancer survival may be improved through smoking cessation, and it is never too late to quit. These results encourage a broader international validation of language models in clinical practice and emphasize that smoking counseling should always be an integral part of cancer care.

DATA SHARING
Patient-level data, even if deidentified or summarized, cannot be transferred without an internal permission procedure according to the Finnish legislation on the secondary use of health data (see 552/2019, www.finlex.fi). Data may be requested from www.auria.fi. The pre-trained ULMFiT language model is available at https://github.com/ AnttiKarlsson/finnish_ulmfit. The pre-trained Finnish BERT model is available at http://turkunlp.org/FinBERT/.