{'fraction_non_alphanumeric': 0.03417038253931002, 'fraction_numerical': 0.021215677071110068, 'mean_word_length': 4.958053691275167, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The COVIDTW study: Clinical predictors of COVID-19 mortality and a novel AI prognostic model using chest X-ray

Background There is a lack of published research on the impact of the first wave of the COVID-19 pandemic in Taiwan. We investigated the mortality risk factors among critically ill patients with COVID-19 in Taiwan during the initial wave. Furthermore, we aim to develop a novel AI mortality prediction model using chest X-ray (CXR) alone. Method We retrospectively reviewed the medical records of patients with COVID-19 at Taipei Tzu Chi Hospital from May 15 to July 15 2021. We enrolled adult patients who received invasive mechanical ventilation. The CXR images of each enrolled patient were divided into 4 categories (1st, pre-ETT, ETT, and WORST). To establish a prediction model, we used the MobilenetV3-Small model with “Imagenet” pretrained weights, followed by high Dropout regularization layers. We trained the model with these data with Five-Fold Cross-Validation to evaluate model performance. Result A total of 64 patients were enrolled. The overall mortality rate was 45%. The median time from symptom onset to intubation was 8 days. Vasopressor use and a higher BRIXIA score on the WORST CXR were associated with an increased risk of mortality. The areas under the curve of the 1st, pre-ETT, ETT, and WORST CXRs by the AI model were 0.87, 0.92, 0.96, and 0.93 respectively. Conclusion The mortality rate of COVID-19 patients who receive invasive mechanical ventilation was high. Septic shock and high BRIXIA score were clinical predictors of mortality. The novel AI mortality prediction model using CXR alone exhibited a high performance.


Introduction
The first coronavirus disease 2019 (COVID-19) outbreak occurred in Wuhan city in China in December 2019. The pathogen of COVID-19 was identified as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). COVID-19 is highly contagious and has rapidly led to a global pandemic. As of December 15, 2021, there were approximately 270 million confirmed cases and 5.3 million confirmed deaths worldwide. 1 Due to effective public health policies, there were only small-scale outbreaks in Taiwan. The total population of Taiwan in 2021 was approximately 23.6 million people. As of December 15, 2021, there were only 16,759 confirmed COVID-19 cases and 849 COVID-19 deaths in Taiwan. 2 The first large-scale COVID-19 outbreak in Taiwan took place between May 15 and July 15, 2021. There were 14,052 new cases of COVID-19 during this period.2 As of May 15, 2021, only 316,200 doses of the COVID-19 vaccine (AstraZeneca) were available in Taiwan.2 Almost all patients with COVID-19 at the initial wave of infection were unvaccinated. Before the advent of the COVID-19 vaccines, a metaanalysis reported that the estimated mortality rate of patients with COVID-19 receiving invasive mechanical ventilation (IMV) was 45%.3 Real-world data are limited in Taiwan. We aimed to investigate the outcomes of patients with COVID-19 receiving IMV in Taiwan.
Scoring systems of CXR are reproducible and reliable tools for predicting the risk of intensive care unit (ICU) admission or mortality among patients with COVID-19.4,5 Such scoring systems include the BRIXIA6 and percent opacification. We assessed whether these scoring could be applied to COVID-19 in Taiwan. Several mortality prediction scores exist, such as the DICE score7 and the 4C Mortality Score.8 These are based on clinical parameters, including sex, age, comorbidities, serum biomarkers, and blood oxygen levels. We aimed to identify such clinical predictors in the Taiwanese population. Artificial intelligence (AI) has recently demonstrated great applicability in medicine. AI models using CXR to prognosticate COVID-19 outcomes are limited. We aimed to build a novel AI prediction model to predict COVID-19 mortality based on CXR alone. Because this study presents data on the first large-scale COVID-19 outbreak in Taiwan, we named the study COVIDTW. The novel AI model was named the COVIDTW model.

Materials and methods
We retrospectively reviewed the medical records of patients with COVID-19 at Taipei Tzu Chi Hospital from May 15 to July 15, 2021. All enrolled patients had reverse transcriptase-polymerase chain reaction (RT-PCR)confirmed COVID-19. We excluded patients who did not receive IMV or were not admitted to the ICU. Patients aged <18 y/o were also excluded. All patients underwent the first CXR (1st CXR) at the emergency department. We extracted baseline characteristics, including age, sex, body mass index (BMI), the first cycle threshold (CT) value of RT-PCR, the ability to perform activities of daily living (dependence or independence), smoking history, educational attainment (cut-point: bachelor's degree or higher), comorbidities, medications, COVID-19 complications, and serum biomarkers, including serum D-dimer level, C-reactive protein (CRP) level, and albumin level.
On the day of endotracheal intubation, we collected data on white blood cell count, CRP level, lactate, blood oxygen level (P/F ratio Z PaO 2 /FiO 2 ), and ventilator setting {positive end respiratory pressure (PEEP) þ pressure control level (PC) Z peak inspiratory pressure (PIP)}. All patients received pressure-controlled ventilation. The maximum PIP was less than 40 cmH 2 O. We calculated the P/ F ratio from the arterial blood gas (1st ABG) obtained about 2 h after endotracheal intubation. The physician and respiratory therapist adjusted the ventilator settings according to the 1st ABG. In the current study, we analyzed the adjusted ventilator settings to predict mortality.
The serial CXR images of each patient were labeled as the 1st CXR (the first CXR in the emergency room), the pre-ETT CXR (the CXR immediately before endotracheal intubation), the ETT CXR (the CXR immediately after endotracheal intubation), and the WORST CXR (the worst CXR during hospitalization). Dr. Chih-Wei Wu (a pulmonologist with 13 years of experience in thoracic radiology) reviewed all CXR images and calculated the BRIXIA and percent opacification scores of each. The BRIXIA score6 is a semiquantitative score and was also calculated by Dr. Yao-Kuang Wu (a pulmonologist with 33 years of experience in thoracic radiology). The mean BRIXIA score was used to predict the mortality. The interclass correlation coefficient (ICC) was used to evaluate the agreement between two experienced experts. We have demonstrated the representative CXR figures of the BRIXIA scores in Supplementary Figs. 1e5. Supplementary Fig. 4 shows the representative WORST CXR of a non-survivor with a high BRIXIA score. Supplementary Fig. 5 shows the representative WORST CXR of a survivor with a relatively low BRIXIA score.
During the COVIDTW study, all patients were unvaccinated, and anti-SARS-CoV-2 monoclonal antibodies were unavailable. Systemic dexamethasone 6 mg QD up to 10 days was routinely administered to all patients receiving IMV according to the proven survival benefits.9 Tocilizumab was routinely administered to patients with serum CRP levels !7.5 mg/dL according to the RECOVERY study.10 Physicians used a combination of midazolam, fentanyl, or cisatracurium to achieve patient-ventilator synchrony. If PaO 2 /FiO 2 < 100, the intensivist would consider prone The natural history of COVID-19 critical illness was presented by a time-to-events table. Day 1 was defined as symptom onset. The events included tracheal intubation, WORST CXR, peak serum CRP level, peak serum D-dimer level, and nadir serum albumin level. We also present other clinical courses, including length of ICU stay, length of hospital stay, and duration of IMV, of deceased and survived patients.
We used Prism 9 statistical software to analyze the data. The ManneWhitney U test was used to compare non-Gaussian continuous variables. Fisher's exact test was used to compare the categorical variables. Logistic regression was used to analyze data with binary outcomes. We utilized the KaplaneMeier method to plot the time-toevent figure and log-rank test to compare the differences. Differences between serial CXR images was tested by oneway analysis of variance (ANOVA). A p value < 0.05 was considered statistically significant.

Experimental setup for artificial intelligence
The small size of the dataset was a major challenge for this research. Deep learning models would easily overfit with the training data, and not perform well with the testing set.
To overcome the data limitation problem, the use of a lightweight model is important. In this work, we used the MobilenetV3-Small model with the "ImageNet" pretrained weights, followed by high Dropout regularization layers specifically to address the overfitting problem. We chose the sigmoid function (Supplementary functional equation 1) as the output activation to achieve binary classification.
Regarding MobileNetV3, in addition to the efficient last stage, the lightweight model introduces a combination of hardware-aware network architecture search (NAS) complemented by the NetAdapt algorithm. Moreover, the network design includes the use of a hard-swish activation and squeeze-and-excitation modules in the "MBConv" blocks. The swish nonlinearity is listed in Supplementary functional equation 2. This has been proven experimentally to improve accuracy. However, as the sigmoid function is computationally expensive, it was modified to produce the hard swish or hswish function (Supplementary functional equation 3).
In addition, we deployed some preprocessing layers within the model, which only applies to the training procedure to conduct data augmentation. These preprocessing layers augment the image data by rotation, translation, flipping, and random contrast adjustment to increase the amount of relevant image data given our limited dataset. Early stopping was also applied to reduce overfitting.
We used the binary cross-entropy as the loss function (Supplementary functional equation 4). Adam was chosen for the optimizer; we first pretrained the model at a learning rate of 0.001 and then fine-tuned it with a slower learning rate of 0.0001.

Data preparation for artificial intelligence
We collected the X-ray images of 64 COVID-19 patients. Each person's data consisted of four X-ray images representing the four groups of diagnostic procedures: 1st CXR, pre-ETT, ETT, and WORST. Our goal was to predict the probability of mortality in each group. In the current research, we performed an experiment with the first three diagnostic groups.
For each state, we trained the model with five-fold cross-validation method to evaluate its performance. The performance was assessed as the average of the five-fold area under the receiver operating characteristic curve (AUROC), accuracy, positive predictive value, sensitivity, and F-1 score. For more details, please refer to the Supplementary section on data preparation for AI.

Ethical statement
The study was approved by the Institutional Review Board of Taipei Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation (Protocol Number: 10-X-045), and the requirement  Data are presented as the number (percentage) or median AE interquartile range. Abbreviations: IQR Z interquartile range, CT Z cycle threshold, RT-PCR Z reverse transcription-polymerase chain reaction, COPD Z chronic obstructive pulmonary disease, ESRD Z end stage renal disease, CRP Z C-reactive protein, ETT-CRP Z the CRP level on the day of endotracheal intubation, CI Z confidence interval, CXR Z chest X-ray, ECMO: extracorporeal membrane oxygenation, 1st CXR Z the first CXR obtained at the emergency department, pre-ETT CXR Z the CXR immediately before endotracheal intubation, ETT CXR Z the CXR immediately after endotracheal intubation, WORST CXR Z the worst CXR during hospitalization. c : The odds ratio approaches 1. a Triple anesthetics means that the patient received midazolam, fentanyl, and cisatracurium simultaneously. b NA (not applicable): the logistic model was not fitted due to complete separation. d If a D-dimer level is > 10000 ng/mL, it is depicted as 10000 ng/mL (detection limit). Because 24 (38%) patients had a peak D-dimer > 10000 ng/mL, peak D-dimer was not included in the multivariate logistic regression. e Denotes statistical significance, which is also marked by gray shading. After univariate logistic regression, the statistically significant risk factors included older age, an education level below a bachelor's degree, a lower P/F ratio, the use of vasopressors, the presence of bacteremia, acute kidney injury, a higher peak serum D-dimer level, a lower nadir serum albumin level, and a higher BRIXIA or percent opacification score on the WORST CXR. The above nine risk factors were chosen for multivariate logistic regression and D-dimer > 10,000 ng/mL was defined as 10,000 ng/mL (detection limit). Because 24 (38%) patients had a peak Ddimer > 10,000 ng/mL, peak D-dimer was not included in the multivariate logistic regression. After multivariate logistic regression, the statistically significant risk factors were vasopressor use and a higher BRIXIA score on the WORST CXR. There were no statistically significant differences in the 1st, pre-ETT, and ETT CXR scores between the two groups. Only one 42-year-old woman received ECMO but unfortunately succumbed to progressive ARDS. The ICC of the 1st, pre-ETT, ETT, and WORST BRIXIA scores were 0.908, 0.820, 0.901, and 0.918, respectively. Both the BRIXIA ( Fig. 2A, P < 0.001) and percent opacification scores (Fig. 2B, P < 0.001) of the WORST CXR were significantly higher among deceased patients. There were no significant differences between the deceased and survived patients in other CXR stages (1st, pre-ETT, and ETT).
There were significant increases in the BRIXIA scores (Fig. 3A) and percent opacification scores (Fig. 3B) from the 1st CXR to pre-ETT, ETT, and WORST CXR.

Discussion
The COVIDTW study presents the first large-scale COVID-19 outbreak in Taiwan. We investigated the outcomes of critically ill patients and clinical predictors of mortality. A novel AI model using CXR alone showed good performance in predicting mortality. Note: Data are presented as the median AE interquartile range or hazard ratio with 95% confidence interval. Abbreviations: IQR Z interquartile range, CI Z confidence interval. a Denotes statistical significance, which is also marked by gray shading. In this study, approximately 50 clinical parameters were extracted. After univariate logistic regression analysis, we identified nine significant risk factors. These nine risk factors were consistent with those reported in the literature. However, after multivariate logistic regression, only two risk factors were identified (use of vasopressors and a higher BRIXIA score on the WORST CXR.) as statistically significant. However, the factors like white blood cell counts, CRP, lactate, blood oxygen level, ventilator settings, and subsequent treatment are known to have a strong impact on mortality, but not in the COVIDTW study. The main reasons for the difference may result from difference in the study population, sample size, sampling time of serum biomarkers, time points of ventilator settings, etc. As for blood oxygen levels, in a retrospective study including 123 mechanically ventilated patients with COVID-19 in China, the P/F ratio on the day of ICU admission was an independent risk factor for mortality.11 In contrast to the abovementioned study, the time point of P/F ratio recording in the COVIDTW study was 2 h after endotracheal ventilation. Continuous changes in lung mechanics after COVID-19 infection are complicated. The most representative time point of the P/F ratio recording requires further investigation.
After endotracheal intubation in patients with COVID-19, few subsequent treatments have been shown to impact mortality. In the COVIDTW study, only patients with serum CRP level >7.5 mg/dL received Tocilizumab according to the mortality benefits from the RECOVERY study.10 However, the optimal CRP cutoff value for mechanically ventilated patients with COVID-19 is unknown. A single-center study that included 154 mechanically ventilated patients with COVID-1912 did not delineate a specific CRP value for enrollment, and the results showed that Tocilizumab was associated with lower mortality. As for ventilator settings, limiting mechanical power13 and use of low tidal volume ventilation14 have mortality benefits for mechanically ventilated COVID-19 patients. However, in the COVIDTW study, the physicians did not calculate the mechanical power owning to limited resources, and there were no uniform protocols for low tidal volume ventilation. Prone positioning reduces mortality rates in moderate-to-severe ARDS due to  In the COVIDTW study, many real-world dilemmas, such as septic shock, acute gastrointestinal bleeding, and severe obesity, impeded routine use of prone positioning.
In the COVIDTW study, the severity of WORST CXR was associated with mortality. Previous studies focused on the first CXR image upon presentation and showed that the BRIXIA and percent opacification scores could predict COVID-19 mortality.4,5 However, in the COVIDTW study, the 1st CXR scores were not significantly different between deceased and survived patients. The reason could be related to the different study populations. In the COVIDTW study, all patients received IMV and were admitted to the ICU, but the ICU admission rates were 8.3% (63/751) 4 and 17% (58/340)5 in prior studies. In addition, in the COVIDTW study, the pre-ETT, and ETT CXR scores also had comparable severities between survived and deceased patients. This suggests that the physicians had similar judgments of respiratory failure and indications for tracheal intubation.
The COVIDTW study suggested that use of vasopressors was a risk factors for mortality. Our results are consistent with those reported in the literature. An observational study enrolling 217 critically-ill COVID-19 patients in the United States reported that vasopressor-requiring shock was significantly associated with mortality. 16 In total, 90% of the deceased patients received vasopressors in contrast to the 54% of patients who survived. A retrospective study including 86 ICU patients with COVID-19 from Saudi Arabia revealed that septic shock was a significant predictor of death (odds ratio Z 58, P < 0.001).17 Consequently, the health care system should focus on hemodynamically unstable patients and prevent complications in patients with COVID-19.
In the early era of the COVID-19 pandemic, the majority of AI studies focused on COVID-19 detection by CXR or chest computed tomography (chest CT). Later, AI-related studies attempted to generate a prognostic model based on chest CT images. However, CXR is more accessible and practical than chest CT in resource-limited health care systems. Few studies have developed AI models based on CXR images for predicting COVID-19 outcomes.18-20 In the United States, Jiao et al. developed AI prediction models using the CXR images and clinical parameters of 1834 patients with COVID-19 to predict critical or noncritical outcome. 18 The AI performance (AUROC) using CXR alone was 0.753. A multicenter study in Italy developed 3 AI models to predict mild or severe outcomes based on the admission CXR images of 820 patients with COVID-19. 19 The AI performance (accuracy) using CXR alone ranged from 0.658 to 0.742. The EXAM model was built by using CXR images and clinical parameters of 16,148 patients. 20 The AUROC for predicting future oxygen requirement was >0.92. The above 3 studies utilized the earliest CXR (at ER or admission) to predict outcomes. However, the COVIDTW model employed 4 different stages of CXR (i.e., 1st CXR, pre-ETT, ETT, and WORST) to prognosticate the outcome. To our knowledge, the COVIDTW model is the first AI-based prediction model built using CXR images of intubated patients. A prognostic model based on the CXR obtained immediately after intubation (ETT CXR in the COVID model) is essential for critically ill patients.

+ MODEL
Transportation of patients to the CT room has several disadvantages, such as catheter dislodgement, lack of oxygen support, and increased transmission risk. Further studies are required to develop reliable AI-based models using postintubation CXR to prognosticate outcomes.
The COVIDTW study has inherent limitations, including its retrospective design and small sample size. Owing to small sample size, the COVIDTW model used five-fold crossvalidation for internal validation. Internal validation usually leads to a higher performance of the prediction model than external validation. Optimally, a novel prognostic model should be externally validated using an independent dataset before incorporation into clinical practice. Further studies are required to enroll more patients with different baseline characteristics to improve the COVIDTW model.
In conclusion, the overall mortality rate of COVID-19 patients receiving IMV was 45%. The risk factors for COVID-19 mortality include the use of vasopressors and a higher BRIXIA score on the WORST CXR. The AI COVIDTW model uses CXR to predict COVID-19 mortality. The models built with the 1st, pre-intubation, post-intubation, and worst CXRs all achieved high performances.