{'fraction_non_alphanumeric': 0.027852728697014537, 'fraction_numerical': 0.01701627643833232, 'mean_word_length': 5.0138707144726515, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Predictive value of the New Zealand Early Warning Score for early mortality in low-acuity patients discharged at scene by paramedics: an observational study

Objectives The utility of New Zealand Early Warning Score (NZEWS) for prediction of adversity in low-acuity patients discharged at scene by paramedics has not been investigated. The objective of this study was to evaluate the association between the NZEWS risk-assessment tool and adverse outcomes of early mortality or ambulance reattendance within 48 hours in low-acuity, prehospital patients not transported by ambulance. Design A retrospective cohort study. Setting Prehospital emergency medical service provided by St John New Zealand over a 2-year period (1 July 2016 through 30 June 2018). Participants 83 171 low-acuity, adult patients who were attended by an ambulance and discharged at scene. Of these, 41 406 had sufficient recorded data to calculate an NZEWS. Primary and secondary outcome(s) and measure(s) Binary logistic regression modelling was used to investigate the association between the NZEWS and adverse outcomes of reattendance within 48 hours, mortality within 2 days, mortality within 7 days and mortality within 30 days. Results An NZEWS greater than 0 was significantly associated with all adverse outcomes studied (p<0.01), compared with the reference group (NZEWS=0). There was a startling correlation between 2-day, 7-day and 30-day mortality and higher early warning scores; the odds of 2-day mortality in patients with an early warning score>10 was 70 times that of those scoring 0 (adjusted OR 70.64, 95% CI: 30.73 to 162.36). The best predictability for adverse outcome was observed for 2-day and 7-day mortality, with moderate area under the receiver operating characteristic curve scores of 0.78 (95% CI: 0.73 to 0.82) and 0.74 (95% CI: 0.71 to 0.77), respectively. Conclusions Adverse outcomes in low-acuity non-transported patients show a significant association with risk prediction by the NZEWS. There was a very high association between large early warning scores and 2-day mortality in this patient group. These findings suggest that NZEWS has significant utility for decision support and improving safety when determining the appropriateness of discharging low-acuity patients at the scene.


INTRODUCTION Background
Early warning scores (EWS) are widely used in hospitals as a tool to aid early recognition of deteriorating patients. The EWS assigns numerical values to elements of a patient's physiological measures such as level of consciousness, blood pressure, heart rate, respiratory rate and oxygen saturation. 1 2 The numerical value can either be aggregated as a total score weighted by the severity of derangement of physiological variables or it may be a single extreme parameter to trigger a rapid system response. 1 2 Several EWS systems are used, including the National Early Warning Score (NEWS), 3 Modified Early Warning Score 4 and VitalPAC EWS. 5 The New Zealand Early Warning Score (NZEWS) was introduced nationwide in 2017 after the need for a standardised system was highlighted in a study by Psirides et al. 6 7 The vital sign (physiological) categories used by the NZEWS are very similar to the United

STRENGTHS AND LIMITATIONS OF THIS STUDY
⇒ This is the first investigation of predictive capability of an early warning score (EWS) in a low-acuity patient cohort and reveals its utility in the prehospital environment. ⇒ The large study cohort (41 406 patients) includes a diverse range of low-acuity clinical presentations. ⇒ No previous studies have investigated the utility of an EWS to evaluate ambulance reattendance. ⇒ This is a retrospective study using clinical data that were not collected for the purpose of calculating an EWS; there may be a selection bias as not all vital sign components were mandatory recordings during the study period. ⇒ We did not capture patients who have self-presented to a medical facility or emergency department.

Open access
Kingdom's Royal College of Physicians NEWS, 3 which is widely used internationally. 8 An updated NEWS (NEWS2) was introduced in 2017 to incorporate peripheral oxygen saturation weighted for the presence of oxygen therapy. 9 The NZEWS system is comparable to the NEWS system. Aggregate vital sign scores in the NZEWS are grouped into five colour-coded categories (white 0, yellow 1-5, orange 6-7, red 8-9 and blue≥10), and a single outlying vital sign measurement can trigger grouping into a highrisk red or blue category. The NEWS has been validated in prehospital settings 10 11 and has demonstrated utility in the emergency medical service (EMS) acute setting for identifying patients transported to hospital who have an increased risk of hospital admission, intensive care treatment or death. 8 12-15 However, there has been little exploration of its use in decision support regarding treatment or transport of patients in low-acuity settings. Low-acuity cases account for the majority (approximately 85%) of the EMS workload in New Zealand. All levels of EMS personnel in New Zealand can autonomously discharge a patient at the scene if immediate medical attention at an emergency department or other medical facility is not indicated. However, these patients who are not transported are a potentially high-risk cohort. 16 Implementation of a tool such as the NZEWS that could appropriately assess the risk of deterioration in these patients would assist in safer decision-making regarding discharge of patients at the scene following EMS assessment.

Objectives
The primary aim of this study was to investigate the association between the aggregated total NZEWS and adverse outcomes in low-acuity adult patients discharged at the scene by EMS. The adverse outcomes evaluated in this study are ambulance reattendance within 48 hours, and mortality within 2, 7 and 30 days. The secondary aim of this study was to determine the accuracy of the NZEWS in predicting an adverse outcome in low-acuity, nontransported adult patients.

Study design
This was a retrospective cohort study of patients who were discharged at the scene by EMS paramedics. Clinical and demographic characteristics were compared between groups of patients categorised by type and severity of adverse outcome. The adverse outcomes investigated were: ambulance reattendance within 48 hours, mortality within 2 days, mortality within 7 days and mortality within 30 days.

Study setting
The patients included in this study are adults attended by St John New Zealand EMS over a 2-year period (1 July 2016 through 30 June 2018).
St John is the largest EMS provider in New Zealand, covering 90% of its population of almost 5 million. The Wellington region of New Zealand is serviced by Wellington Free Ambulance. The St John New Zealand EMS has previously been described in detail. 17 All paramedic practice levels in New Zealand can autonomously determine whether ambulance transport is required or not and recommend the most appropriate medical facility to which the patient should be transported. EMS personnel can also recommend that the patient takes private transport to a medical facility. Currently in New Zealand, EMS personnel practicing at the level of paramedic and above are registered health professionals.
The NZEWS is not currently included in the Clinical Procedures and Guidelines followed by the New Zealand ambulance services.

Participants
The selection of these low-acuity, discharged-at-scene cases has been described in detail previously. 17 Briefly, clinical triage status codes are given to a patient by NZEMS personnel. The NZEMS status codes are: 0-dead, 1-immediate threat to life, 2-potential threat to life, 3-unlikely threat to life, 4-no threat to life. For this study, patients with a final status code of 3 or 4 were deemed low-acuity and appropriate for non-transport. Inclusion criteria included: adults (15 years or older), National Health Index (NHI) identifier recorded, low acuity (final status 3 or 4) and discharged at scene by an attending paramedic. Exclusion criteria included: no recorded NHI number, documented evidence of ambulance transport, under palliative care, high acuity (final status 1 or 2), death prior to ambulance personnel leaving the scene (status 0) and frequent users of the ambulance service (≥3 ambulance attendances within a month or ≥12 ambulance visits in a 2-year period) (see figure 1).

Patient and public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Variables
This study was conducted using St John New Zealand electronic patient report form (ePRF) data, collected for the purposes of clinical audit. The outcome variables of interest were ambulance reattendance within 48 hours of the index event, 2-day, 7-day and 30-day mortality. Ambulance reattendance was identified by encrypted NHI matching within the ePRF dataset. Data were only included for the first ambulance attendance for those patients who had an ambulance reattend within 48 hours, and subsequent reattendance within the 48 hours was noted as an adverse outcome. Only the first ambulance attendance for each patient within the 2-year study period was included in the analysis. The data variables identified included respiration rate, oxygen saturation, use of supplemental oxygen, temperature, systolic blood Open access pressure, heart rate and level of consciousness (alert, voice, pain, unresponsive), and these were used to retrospectively calculate the NZEWS. 7 Only patients with a complete set of all seven vital signs required for the calculation of an NZEWS were included in the analysis. In the case where multiple vital sign measures were recorded, only the last recorded measurement was included in the NZEWS calculation. The total aggregate NZEWS are grouped into five deterioration risk groups (from low risk to high risk): white (NZEWS=0), yellow (NZEWS 1-5), orange (NZEWS 6-7), red (NZEWS 8-9 or any vital sign in red zone) and blue (NZEWS≥10 or any vital sign in blue zone) (figure 2).
Demographic variables collected from ePRF have been extensively described previously. 17 The date of death was provided by the Ministry of Health, New Zealand, matched by patient NHI.

Statistical methods
Logistic regression was used to investigate outcome differences, with data presented as adjusted OR (AOR) with 95% CI. For the unadjusted logistic regression, bivariate associations were examined between the dependent and each independent variable. For multivariate logistic regression, the variables age, sex, ethnicity, deprivation and NZEWS were included in a forward conditional model.
The area under the receiver operating characteristic curve (AUROC) was calculated to evaluate predictive ability of the total aggregate NZEWS with each adverse outcome, along with its 95% CI. We computed sensitivity and specificity for each outcome using a cut-off value that generated the greatest Youden's index. 20 An unmodified, non-weighted Youden's index was calculated in SPSS (V.28.0) by determining the maximal value of sensitivi-ty+specificity -1.

RESULTS
Following application of the exclusion criteria, 83 171 low-acuity, adult patients were discharged at scene during this 2-year period, with 71 580 first ambulance attendance events. Of these, 41 406 (58%) had sufficient data to calculate an NZEWS (figure 1). The characteristics of the NZEWS calculable subset were similar to the lowacuity, non-transported subset without calculable NZEWS (table 1).
Most of the cohort (94.4%) were in the low risk of deterioration groups with an NZEWS of ≤5 (white,  Open access n=20 438 and yellow, n=18 615) (table 2, figure 2). In total, 5.5% of the cohort were in the high-risk of deterioration groups (red, n=1956 and blue, n=338). The smallest NZEWS group was orange (n=59), as 314 patients within this cohort were upgraded by having a single vital sign measurement within the red or blue thresholds (table 1, . As a result of this small cohort size, there were very few cases in the orange NZEWS group with an adverse outcome, and no orange cases in the 2-day mortality cohort (table 2). The distribution of patients with adverse outcomes by NZEWS colour grouping is shown in table 2. Open access using a cut-off value of 1.50 (NZEWS≥2) corresponding to the greatest Youden's index of 0.34. The NZEWS appears to have the most utility for predicting early mortality at 2 days, with its utility to predict mortality decreasing over time from the index ambulance attendance. The AUROC score for 48-hour ambulance reattendance of 0.55 (95% CI: 0.53 to 0.56) revealed that NZEWS has no discriminatory ability for reattendance, yielding a sensitivity of 0.29 and a specificity of 0.78 when using a cut-off value of 1.50 (NZEWS≥2) corresponding to the greatest Youden's index of 0.07.

DISCUSSION
In this large study of low-acuity, non-transported patients, we have investigated the utility of using an EWS systemthe NZEWS-to predict the adverse outcomes of ambulance reattendance within 48 hours, and mortality at 2, 7 and 30 days. All NZEWS>0 categories were associated with increased odds of patient mortality. The strongest association was identified with 2-day mortality, where the most at-risk of deterioration (blue) group had over 70 times the odds of dying than those with an NZEWS of 0. The NZEWS showed a degree of accuracy for all mortality outcomes. The predictability scores for mortality at 2 and 7 days post-ambulance attendance indicated that the NZEWS had moderate accuracy in identifying patients at risk of dying, while NZEWS provided lower accuracy in identifying patients at risk of dying within 30 days. Our findings indicate that the NZEWS is not a good predictor of ambulance reattendance within 48 hours.
To date, almost all studies evaluating the use of a prehospital EWS have focused on acutely unwell patients. Patients who are not transported to hospital represent a high-risk population for the EMSs. We have previously shown that the vast majority (95%) of patients autonomously discharged at scene by New Zealand ambulance personnel do not require ambulance reattendance within 48 hours of the initial examination. 17 The combination of the accuracy of prediction and the regression analysis indicate that the NZEWS has significant utility in identifying patients with an increased risk of mortality, especially within 2 days of the initial attendance. However, the NZEWS may not be quite as useful for identifying patients likely to require a subsequent ambulance attendance. The finding of a better predictive value for NZEWS with earlier mortality (2 days) is also consistent with other studies that have evaluated NEWS2 in both the prehospital and in-hospital settings. 13 15 21-24 While death may not have been unexpected in some of these patients, it is important to note that identifiable palliative cases were excluded from this cohort.
An EWS system does not replace a thorough clinical evaluation of the patient. Patients with high NZEWS scores who did not merit transport to hospital included cases of simple hyperventilation, rapidly responding hypoglycaemic episodes in diabetic patients, and selflimiting seizures in diagnosed epileptics. The NZEWS provides an additional signal of the risk of patient deterioration to the evaluating clinician. Integration of the NZEWS into the electronic platforms currently used to record patient assessment and clinical information would provide a quick tool for assessing risk of deterioration. If all patients with an NZEWS>5 were classified as high-risk and mandated for transport in NZEMS guidelines, 5.7% of the cohort in this study would have been transported to a medical facility. Given that around 20% of all the ~461 000 patients that St John New Zealand attends are not transported, 17 applying this theoretical mandate for transport based on the NZEWS score would have resulted in around 5250 more ambulance transports to an emergency department or other medical facility each yearapproximately 14 more patients per day. The introduction of the NZEWS to improve patient safety in discharged at scene cases should be considered, as it is unlikely that these ~14 patients per day nationally would overburden emergency departments. Any implementation of NZEWS in the prehospital setting should be accompanied by a study, pre-implementation versus post-implementation, to directly evaluate the impact on patient outcomes. In New Zealand, a competent patient can choose not to be transported to a medical facility for further care, despite an EMS recommendation. A quarter of patients in the red and blue NZEWS groups declined transport.
There may be other changes that could improve the predictability of the NZEWS. Currently, the patient's physiological readings considered as derangement from normal are irrespective of age or ethnicity. It may be that specific factors relating to patient demographics and a more tailored EWS could further improve the utility and accuracy of prediction. Potential for tailoring the EWS to age has been demonstrated in both the prehospital and in-hospital settings, indicating that the addition of age Open access as a parameter to the NEWS has improved prediction of short-term mortality. 25 26 Future research into the manipulation of the NZEWS colour categories (eg, altering colour thresholds) may also improve the accuracy for this low-acuity cohort.
Tailoring the NZEWS for ethnicity may also be needed. In the New Zealand setting, there are significant health inequities between ethnic groups, particularly affecting New Zealand Māori (indigenous population) and Pacific peoples. For example, life expectancy for Māori is more than 7 years lower than European New Zealanders, thought clinically to be related to increased comorbidities. Māori experience two times the rate of avoidable mortality compared with non-Māori. 27 One New Zealand study has suggested that the settings of the NZEWS provide too high a threshold for older people, for those with multimorbidities, and for Māori and Pacific peoples, 28 meaning some within these populations are not identified by the NZEWS, perhaps leading to delayed interventions. Evaluation of the relationship between ethnicity and the NZEWS vital sign components in the prehospital environment will be important to ensure equitable and culturally appropriate care. The cultural needs of Māori patients should also be a consideration when a new prehospital pathway is introduced. While introduction of a mandated NZEWS system may be clinically justified, it is important to also consider a holistic view of health which includes physical, mental, spiritual and family health as outlined in the Māori health model-Te Whare Tapa Whā. 29 Our study is limited to data that have been collected primarily for the purposes of transfer of patient care and clinical audit. The NZEWS was calculated retrospectively from the last (or only) vital sign measurements recorded. During this study period, the clinical procedures and guidelines used by NZ paramedics did not mandate recording of all vital sign components required to calculate NZEWS in discharged-at-scene patients. This may have resulted in a selection bias in the cohort, as patients without a full set of vital signs were excluded from this study. The outcomes in this study were limited to ambulance reattendance within 48 hours and mortality at 2, 7 and 30 days post attendance. Discharged-at-scene patients who self-presented at a medical facility or hospital emergency department were not captured in this study. We have not optimised the cut-off values of the NZEWS, as we wanted to address the utility of implementing the current NZEWS system in a prehospital context. Further research into optimising the NZEWS for the prehospital setting by including factors such as age and ethnicity is warranted and is the focus of further research.

CONCLUSIONS
Adverse outcomes in low-acuity, non-transported patients show a significant association with risk prediction by the NZEWS. The very high association between high-risk EWS and 2-day mortality suggests that this has significant utility in the setting of decision support when determining the appropriateness of discharging low-acuity patients at the scene. Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests VFT, AB, TS and BD are employees of St John New Zealand. AS is an employee of Wellington Free Ambulance New Zealand.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not applicable.
Ethics approval This study involves human participants and was approved by Auckland University of Technology Ethics Committee (18/231). Due to the large number of patients involved, and the low risk of patient recognition from the deidentified data, consent was not obtained from each participant in the study.
Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement No data are available.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Open access This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/. Supplementary Figures 1 and 2 Supplementary Figure 1. Primary clinical impressions from EMS record of low-acuity, dischargedat-scene patients with a Red NZEWS. Figure 2. Primary clinical impressions from EMS record of low-acuity, dischargedat-scene patients with a Blue NZEWS.

Supplementary
Metabolic includes (but is not limited to): epilepsy, seizure, hypoglycaemia, confusion Miscellaneous includes (but is not limited to): generally unwell, decreased mobility, dehydration, diarrhoea and vomiting, palliative care, lethargy.