{'fraction_non_alphanumeric': 0.0322710230316015, 'fraction_numerical': 0.012006784502767364, 'mean_word_length': 5.71210305572199, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Respiratory support status from EHR data for adult population: classification, heuristics, and usage in predictive modeling

Abstract Objective Respiratory support status is critical in understanding patient status, but electronic health record data are often scattered, incomplete, and contradictory. Further, there has been limited work on standardizing representations for respiratory support. The objective of this work was to (1) propose a practical terminology system for respiratory support methods; (2) develop (meta-)heuristics for constructing respiratory support episodes; and (3) evaluate the utility of respiratory support information for mortality prediction. Materials and Methods All analyses were performed using electronic health record data of COVID-19-tested, emergency department-admit, adult patients at a large, Midwestern healthcare system between March 1, 2020 and April 1, 2021. Logistic regression and XGBoost models were trained with and without respiratory support information, and performance metrics were compared. Importance of respiratory-support-based features was explored using absolute coefficient values for logistic regression and SHapley Additive exPlanations values for the XGBoost model. Results The proposed terminology system for respiratory support methods is as follows: Low-Flow Oxygen Therapy (LFOT), High-Flow Oxygen Therapy (HFOT), Non-Invasive Mechanical Ventilation (NIMV), Invasive Mechanical Ventilation (IMV), and ExtraCorporeal Membrane Oxygenation (ECMO). The addition of respiratory support information significantly improved mortality prediction (logistic regression area under receiver operating characteristic curve, median [IQR] from 0.855 [0.852—0.855] to 0.881 [0.876—0.884]; area under precision recall curve from 0.262 [0.245—0.268] to 0.319 [0.313—0.325], both P < 0.01). The proposed generalizable, interpretable, and episodic representation had commensurate performance compared to alternate representations despite loss of granularity. Respiratory support features were among the most important in both models. Conclusion Respiratory support information is critical in understanding patient status and can facilitate downstream analyses.


INTRODUCTION
Managing respiratory status and providing appropriate respiratory support to prevent or mitigate hypoxemia is a critical aspect of clinical management, especially for patients suffering from respiratory conditions such as coronavirus disease 2019 . 1 Leveraging respiratory support information not only provides a more complete clinical picture, but it can also be used in downstream analyses such as sub-phenotyping or predictive modeling as a source of features or to identify endpoints. 2,3 However, generating generalizable conclusions from respiratory support information is difficult due to the heterogeneity of respiratory support methods and settings, the lack of standardized representations, and the poor quality of data regarding respiratory support. 4 Standardization of patient data representation has accelerated knowledge discovery, replication of results, and translation into practice. While many parts of healthcare information have been standardized-such as the International Classification of Diseases (ICD) for diagnoses, LOINC for measurements and observations, or the all-encompassing UMLS-much of the patient information in EHR data, such as clinical cultures and respiratory support methods, remains unmapped. 5,6 Standardization of respiratory support is made challenging due to the inherent diversity of methods, especially when considering methods designed for subpopulations, variations, and modifiable settings. Worse yet, the lack of a widely adopted standardization schema has resulted in the usage of highly heterogeneous terms in literature and practice, even for identical conceptsfor example, heated and humidified high-flow nasal cannula has been described as high-flow nasal cannula, high humidity nasal cannula, high-flow nasal oxygen therapy, or referred to by brand names such as Airvo TM or Optiflow TM . [7][8][9] Prior efforts to standardize respiratory support terms resulted in high granularity, which while necessary for coverage, can be excessive for downstream tasks. For example, the "Respiratory Therapy" concept in SNOMED-CT contains 14 children, of which "Oxygen Therapy" itself contains 14 children. 10 Thus, there is a need for a pragmatic and parsimonious standardization of respiratory support terms.
Beyond the lack of standardization, extraction of respiratory support status from electronic health records (EHRs) data is made challenging due to scattered, incomplete, and often contradictory documentation which necessitates the usage of auxiliary documentation and heuristics to determine respiratory support status. [11][12][13] Prior heuristic development work, however, are dataset-specific and focus on endotracheal intubation thus fail to address other strata of respiratory support. 11,12 Therefore, the objective of the study was to (1) propose a preliminary, parsimonious, and pragmatic terminology system for respiratory support stratified by severity of hypoxemia; (2) develop (meta-)heuristics for the construction of respiratory support episodes from raw and heterogeneous EHR data; and (3) evaluate the terminology system and heuristics by measuring its impact on 30-day mortality prediction through the assessment of feature importance and feature ablation studies.

MATERIALS AND METHODS
Study design, data sources, and population All patients 18 years of age admitted to all hospitals within a large Midwestern healthcare system serving the metropolitan St. Louis, mid-Missouri, and southern Illinois regions between March 1, 2020 and April 1, 2021 were eligible for inclusion. Patients were included if they had a COVID-19 PCR or antigen test, positive or negative, within 14 days prior to or 7 days after hospital admission. Patients were excluded if they had a hospital length of stay (LOS) < 24 h to allow for a sufficient observation window for the predictive modeling study. Patients without associated demographics, comorbidities, or location data were excluded. Only patients with at least 5 heart rate and 5 SpO 2 measurements during the first 24 h of hospital arrival were included. EHR data, including demographics, vital signs, lab results, flowsheet entries, and so on, were extracted for all included subjects. This project was approved with a waiver of informed consent by the Washington University in St. Louis Institutional Review Board.

Classification system of respiratory support methods
To facilitate generalizability and reproducibility of studies leveraging respiratory support information, we propose the following terminologies, in increasing severity: Low-Flow Oxygen Therapy (LFOT), High-Flow Oxygen Therapy (HFOT), Non-Invasive Mechanical Ventilation (NIMV), Invasive Mechanical Ventilation (IMV), and ExtraCorporeal Membrane Oxygenation (ECMO). 11,14 Meta-heuristics for identification of respiratory support episodes The authors of MIMIC-III, in addition to the data, also published code for data processing and analysis to expedite and encourage collaborative research. 11 Among the SQL scripts in their GitHub repository is one for calculating mechanical ventilation duration. Essentially, their logic was to chain together proximal pieces of documentation that are indicative of mechanical ventilation to form episodes with start and end times. Their heuristic has since been used successfully by researchers using the MIMIC-III dataset. 13 To generalize the heuristic for other forms of respiratory support beyond mechanical ventilation, and for other datasets beyond MIMIC-III, we developed a generalized version of the MIMIC-III heuristic-a "meta-heuristic"-to guide the development of heuristics for the assembly of respiratory support episodes as follows: 1. Define 2 parameters: a. MIN_DURATION for the minimum episode duration b. EXTENSION_TOLERANCE for the maximum allowable time gap between documentation for the formation of episodes 2. Identify timestamped documentation that are indicative of the presence of respiratory support 3. Link consecutive documentation occurring within EXTEN-SION_TOLERANCE into episodes 4. Discard any episodes with duration less than or equal to MIN_DURATION Next, as we conceived of the respiratory support methods as being mutually exclusive, respiratory support episodes are "flattened" into a single timeline such that at any given time, a patient is on either no respiratory support or a single respiratory support method, by giving higher severity methods priority. Finally, the respiratory support trajectories are "repaired" such that gaps between episodes with a duration less than EXTENSION_TOLERANCE are filled by extending the preceding episode. Also, gaps at the beginning and end of the patient stay (between encounter start time and first respiratory support episode start time, and between last respiratory sup-port episode end time and encounter end time), if less than MIN_DURATION, are filled by extending the first or last episode, respectively.

Evaluation through in-hospital mortality prediction
A predictive modeling study was designed to evaluate the utility of the respiratory support information extracted through our heuristics on downstream analyses. The task was to predict in-hospital mortality within 30 days, at 24 h after hospital arrival for a COVID-19tested, adult cohort presenting to an emergency department (ED); 121 baseline features were generated from demographic, laboratory, vital sign, and other clinical data extracted from the EHR. For most numeric measurements, the median value during the observation window was extracted, but for frequent measurements such as heart rate, other distributional statistics (25th quantile, 75th quantile, and interquartile range) were also extracted.
Ten additional, respiratory-support-derived features were generated using the proposed classification schema and heuristics which included duration of respiratory support per type, and the last respiratory support during the observation period. For comparison, we identified a small set of measurements related to respiratory status: fraction of inspired oxygen (FiO 2 ) and oxygen flow rate. We also extracted the EHR-native representation of respiratory support called oxygen delivery method (O 2 Del Method) which included ETT, CPAP, T-Piece, and so on. Lastly, we also considered a set of features based on the proposed classification, but using the raw time-stamped data prior to assembly into episodes. The feature sets including explicit respiratory support information-O 2 Del Method, Raw, and Proposed-all also include "Baseline" and "Related" features (Online Supplemental Materials, eTable 1, eFigure 1).
The compared algorithms were logistic regression (LogReg) and XGBoost (XGB). For LogReg, features were standardized and mean-imputed whereas for XGB, features were left as is. Hyperparameters such as regularization strength were optimized for log loss using the baseline features through 1000-iteration, 4-fold crossvalidation (Online Supplemental Materials, eAppendix A). Once optimal hyperparameters were identified, five replicates of 2-fold cross-validation was performed to generate a distribution of performance metrics: area under receiver operating characteristic curve (AUROC), area under precision recall curve (AUPRC), and negative log loss. 15 The distributions of performance metrics were compared using the Wilcoxon signed-rank test (2-way, paired). 16 Feature importance was quantified using SHapley Additive exPlanations (SHAP) values for the XGBoost model, and coefficient values for the logistic regression model, both of which were aggregated over 100 bootstrap samples. 17,18  Documentation of "NPPV Status" as "In Use" in the "Adult NPPV/NIV" flowsheet Invasive Mechanical Ventilation (IMV) Documentation of "Vent Status" as "In Use" in the "Ventilator Documentation" flowsheet ExtraCorporeal Membrane Oxygenation (ECMO) Documentation of "Pump Flow (L/min)" or "ECMO Pump Speed (RPM)" in the ECMO or VAD flowsheets

Statistical analysis
Variables were summarized using frequencies and proportions for categorical data and medians and interquartile ranges or means and standard deviations for continuous data. Statistical comparisons were performed using the Chi-square and Mann-Whitney U tests where appropriate unless specified otherwise. A P-value < 0.01 was considered statistically significant. All resampling analyses, crossvalidation and bootstrap, were performed using a fixed seed. All analysis and figure generation were performed with Python version 3.7.1 (Python Software Foundation, Beaverton, OR) using the following packages: scipy, numpy, pandas, matplotlib, sklearn, xgboost, and shap. 17,19-24

RESULTS
The severity-stratified respiratory support methods and the documentation serving as evidence for each method are listed in Table 1.
Examples of the full heuristic application process, along with documentation germane to respiratory status, can be found in Figure 1. Figure 2 shows respiratory support utilization over time, with patients temporally aligned at ED arrival. Cohort characteristics and outcomes for the patient population used in this study can be seen in Table 2. During the study period there were 45 908 hospitalizations lasting at least 24 h available for analysis. Of these, 1601 (3.5%) experienced in-hospital death within 30 days. Non-survivors were older, more likely to be male, more likely to be COVID-19 positive, and have a longer length of stay ( Table 2).
The optimized hyperparameters (Online Supplemental Materials, eAppendix A) were used for generating distributions of performance metrics through repeated cross-validation ( Figure 3). For both XGB and LogReg, the addition of "Related" features significantly improved on "Baseline," and the addition of "O 2 Del Method," "Raw," or "Proposed" features improved on "Related" across all 3 metrics: AUROC, AUPRC, and negative log loss (Figure 3, Online Supplemental Materials, eTable 2, eTable 3). However, "O 2 Del Method," "Raw," and "Proposed" rarely differed significantly, and when they did, the differences were very small as was the case for LogReg AUROC between "O 2 Del Method" and "Proposed" Six of the top 20 most impactful features for the LogReg model were respiratory-support-derived features, including last respiratory support, IMV, and LFOT duration (Figure 4). For the XGB model, respiratory-support-derived features ranked 10th (last respiratory support, IMV) and 17th (LFOT duration) ( Figure 5).

DISCUSSION
In this study, we (1) propose a preliminary, parsimonious, and pragmatic terminology system for respiratory support methods; (2) develop (meta-)heuristics for extraction of respiratory support information from EHR data; and (3) investigate the utility of the respiratory support information extracted through the proposed terminology system and heuristics via a mortality prediction study in a COVID-19-tested, ED-admit, adult cohort. The developed heuristic  Comparison of variables between those with and without the primary outcome of 30-day in-hospital mortality was performed using Mann-Whitney U test for continuous variables, and v 2 for categorical variables. Statistical significance, *P < 0.01, is indicated by.
was successfully applied to EHR data to extract respiratory support episodes, which were then used for in-hospital mortality prediction as features, which were found to be among the most important features for both LogReg and XGB models. Compared to models using demographics and commonly documented lab results and vital signs, the addition of respiratory-supportrelated information, FiO 2 and O 2 flow rate, significantly improved prediction performance for both XGB and LogReg across all measured performance metrics. Moreover, the additional inclusion of explicit respiratory support information further improved performance significantly, again for both model types and across all metrics.
Because there are no other dataset-agnostic, full-severity-spanning classification, and heuristic system for respiratory support information found in literature, we compared our proposed approach against 2 other methods of explicit respiratory status representation: "O 2 Del Method" which used the EHR-native representation and "Raw" which uses the data elements for the proposed approach but prior to assembly into episodes (Online Supplemental Materials, eTable 2, eFigure 1). The proposed representation had commensurate performance to alternate representations of explicit respiratory support information, despite loss of both conceptual and temporal granularity resulting from the aggregation of heterogeneous timestamped raw data into the more human-understandable format of encounter-spanning series of episodes (Figure 3). The increase in model performance associated with the addition of explicit respiratory support information in XGB was less than that of LogReg. We hypothesize that more complex models are able to infer respiratory support status or reconstitute information contained in respiratory status based on other features.
For the models using features from the proposed approach, IMV status and duration was an important predictor which is unsurprising-patients who are intubated are known to have higher rates of in-hospital mortality. 25 This result simply underscores the importance of leveraging respiratory support information, especially those of high severity, for understanding patient status. However, even features based on low-flow oxygen therapy status were among the most important features for both the XGB and LogReg models, indicating that lower severity respiratory support information is also critical for developing a complete clinical picture of patients.
In this study, respiratory support information was used as features for in-hospital mortality prediction. However, there are many other potential uses of respiratory support information, such as endpoint identification, patient sub-phenotype discovery, patient trajectory analytics, or characterization of patient cohorts. 13,14,26 As documentation regarding respiratory support status varies across time and across sites, identification of timestamped documentation that serve as evidence for respiratory support cannot be generalized and must be specified in each study by researchers with Figure 3. In-hospital mortality prediction feature ablation performance comparison. Comparison of in-hospital mortality prediction performance for LogReg and XGB models with varying sets of features from 5-repeat, 2-fold cross-validation. "Baseline" includes demographics, common lab results, and vital signs from the EHR data. "Related" also includes O 2 flow rate and fraction of inspired oxygen. In addition, "O 2 Del Method" includes the EHR-native representation of respiratory support status, "Raw" includes data from the proposed approach prior to assembly into episodes, and "Proposed" includes features derived from respiratory support episodes based on the proposed approach. The center horizontal line represents median, box represents the interquartile range between 25th and 75th percentiles, and whiskers represent 2.5th and 97.5th percentiles. Abbreviations: AUPRC: area under precision recall curve; AUROC: area under receiver operating characteristic curve; LogReg: logistic regression; XGB: extreme gradient boosted trees model. appropriate knowledge of local practice patterns contained within the dataset. Therefore, a (meta-)heuristic was developed which provides the structure for developing heuristics to establish episodes for any respiratory support method.
As is typical of studies using EHR data, this study suffers from missingness and inaccuracy of information. For example, we identified patients admitted through the ED with a recent positive COVID-19 test who were transferred to and remained in the ICU for several days, yet had no documentation of respiratory support throughout their entire stay. While heuristics can work with scattered, conflicting, and incomplete documentation, significant missingness will still result in unrealistic scenarios. There is trade-off in setting the MIN_DURATION parameter-if it is too long then temporary/interim respiratory supports will be underrepresented; conversely, if it is too short then the heuristic will allow for unrealistically rapid oscillation among respiratory support methods. Additionally, tracheostomy status was considered orthogonal to the proposed system. NIMV is often used nightly for sleep apnea; thus, researchers utilizing the heuristic must decide whether to ignore those episodes based on their needs. Also, patients can be connected to a device for respiratory support, but not be actively using them (eg, delivering no or low-flow oxygen through a device capable of delivering high-flow oxygen), thus researchers must decide which is more important for their work: the occupation of the device or the active use of the device. Respiratory support methods are ever-evolving-helmet NIV and high-flow nasal cannula, for instance, have only recently been used for adult patients, meaning that these terminology systems will also require regular revisiting and updating. 27

CONCLUSION
To facilitate generalizable and reproducible research, a terminology system was developed for standardized representation of respiratory support methods. (Meta-)heuristics were also developed to enable extraction of respiratory support episodes from EHR data, and transformation into encounter-spanning set of respiratory support trajectories. To demonstrate the utility of respiratory support information extracted through proposed methods, feature ablation and feature importance analyses were performed via an in-hospital mortality prediction study for COVID-19-tested, ED-admit, adult patients. The addition of features generated from the proposed approach significantly improved model performance. Further, those features were found to be among the most important for models. Finally, the proposed approach, which generated more interpretable and generalizable representations, despite the loss of conceptual and temporal granularity, had commensurate performance to alternate representations of explicit respiratory support information.