{'fraction_non_alphanumeric': 0.03001334728184409, 'fraction_numerical': 0.020598102521554057, 'mean_word_length': 5.27478497057492, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Do paediatric early warning systems reduce mortality and critical deterioration events among children? A systematic review and meta-analysis

Aim We conducted a systematic review and meta-analysis to answer the question: Does the implementation of Paediatric Early Warning Systems (PEWS) in the hospital setting reduce mortality, cardiopulmonary arrests, unplanned codes and critical deterioration events among children, as compared to usual care without PEWS? Methods We conducted a comprehensive search using Medline, EMBASE, Cochrane Central Register of Controlled Trials, Cumulative Index to Nursing and Allied Health Literature and Web of Science. We included studies published between January 2006 and April 2022 on children <18 years old performed in inpatient units and emergency departments, and compared patient populations with PEWS to those without PEWS. We excluded studies without a comparator, case control studies, systematic reviews, and studies published in non-English languages. We employed a random effects meta-analysis and synthesised the risk and rate ratios from individual studies. We used the Scottish Intercollegiate Guidelines Network (SIGN) to appraise the risk of bias. Results Among 911 articles screened, 15 were included for descriptive analysis. Fourteen of the 15 studies were pre- versus post-implementation studies and one was a multi-centre cluster randomised controlled trial (RCT). Among 10 studies (580,604 hospital admissions) analysed for mortality, we found an increased risk (pooled RR 1.18, 95% CI 1.01–1.38, p = 0.036) in the group without PEWS compared to the group with PEWS. The sensitivity analysis performed without the RCT (436,065 hospital admissions) showed a non-significant relationship (pooled RR 1.17, 95% CI 0.98–1.40, p = 0.087). Among four studies (168,544 hospital admissions) analysed for unplanned code events, there was an increased risk in the group without PEWS (pooled RR 1.73, 95%CI 1.01–2.96, p = 0.046) There were no differences in the rate of cardiopulmonary arrests or critical deterioration events between groups. Our findings were limited by potential confounders and imprecision among included studies. Conclusions Healthcare systems that implemented PEWS were associated with reduced mortality and code rates. We recognise that these gains vary depending on resource availability and efferent response systems. PROSPERO registration: CRD42021269579.


Introduction
Paediatric Early Warning Systems (PEWS) were derived to meet the need for early and accurate identification of children at risk of clinical deterioration. 1 Published PEWS use a constellation of physiological criteria (including vital signs), clinical assessment findings, preexisting medical conditions, and clinician or parental concerns. 2,3 Elevated scores are an early surrogate for cardiopulmonary collapse, and PEWS are reported to forewarn by up to 11 hours before actual deterioration. 3,4 Intuitively, implementation of PEWS creates a formalised structure within healthcare systems that may detect early changes in clinical status. PEWS are a component of the "afferent" arm of a Rapid Response System that then informs and facilitates interventions performed by a rapid response team (RRT) or a paediatric medical emergency team (MET) (the "efferent" arm). Elevated PEWS were found to correlate with the severity of illness. 5 Adding PEWS to existing emergency department (ED) systems improved triage in 5 diverse EDs across 4 European countries. 6,7 Others reported that PEWS implementation resulted in good data capture and staff satisfaction. 8,9 Others have questioned the true value of PEWS implementation for patient care. In a large validation study in the Netherlands, they found that none of the PEWS had high sensitivity and specificity. 10 Although elevated scores were associated with increased likelihood of paediatric intensive care unit (PICU) readmission, thresholds were not sufficiently sensitive nor specific to be clinically useful. 11 The Effect of a Pediatric Early Warning System on All-Cause Mortality in Hospitalized Pediatric Patients (EPOCH) Randomized Clinical Trial across 21 hospitals in 7 countries showed that implementing PEWS did not decrease all-cause mortality. 12 The literature on effectiveness of PEWS varies in the location of PEWS implementation (general paediatric units, specific specialist wards or EDs), study populations (general paediatric versus specific subpopulations), associated interventions, as well as the choice of outcome measures. [13][14][15] The impact of PEWS on clinicallyimportant patient-centric outcomes may be confounded by the presence of co-interventions present in many before-after study designs. 16 In addition, mortality as a critical outcome is rare, especially in resource-rich healthcare facilities. Assessment of PEWS effectiveness based on mortality may underestimate the benefits of PEWS in the overall care of acutely ill children.
In view of the above contention, we therefore sought to perform an updated systematic review and meta-analysis, from January 2006 to April 2022 to answer the question: Does the implementation of PEWS (with or without RRT/PMET) in the hospital setting reduce mortality, cardiopulmonary arrests, unplanned codes and critical deterioration events among children, as compared to usual care without PEWS?

Methods
This systematic review was registered with PROSPERO (CRD42021269579). We extended the duration of included studies, initially from January 2011 to December 2020, to 2006 to April 2022 to present a comprehensive and updated review of the literature. All items were reported in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines. 17

Eligibility criteria
We included all studies with children <18 years old performed in the inpatient units and EDs of paediatric hospitals but excluded outpatient clinics. We included studies published between January 2006 and April 2022 on the general paediatric population as well as those that focused on specific populations (e.g. oncology or cardiology units). We excluded patients >18 years and preterm infants. We compared patient populations among whom PEWS were implemented (intervention) to those among whom PEWS were not implemented (comparator). The implemented PEWS could include any formalised set of criteria to alert clinicians to potential deterioration. We excluded all studies that did not have a comparator group (no PEWS). For outcomes, we included: (1) mortality; (2) cardiopulmonary arrest; (3) unplanned code events; and (4) critical deterioration. We chose the specific outcomes of cardiopulmonary arrests and unplanned codes because these were individually measured and accounted for in the literature. Under critical deterioration, we included the following composite definition of significant clinical deterioration: (a) Unplanned/ crash tracheal intubation; (b) Unanticipated fluid resuscitation and inotropic/vasopressor use; (c) Cardiopulmonary resuscitation (CPR) or Extracorporeal Membrane Oxygenation (ECMO); and (d) Death in patients (all-cause mortality) without a Do Not Resuscitate (DNR) order. 18 We also included unplanned or emergency admissions to Paediatric Intensive Care Unit (PICU) as these reflected the utilisation of critical care resources. For study designs, we included randomised controlled trials (RCTs) and non-randomised studies including before-and after-implementation studies. Case series, case control, systematic reviews, unpublished studies and studies published in non-English languages, were excluded.

Information sources
We included the following electronic databases:

Search strategy
We drew up a comprehensive search strategy with inputs from a medical librarian as well as our clinical team. We used and exploded the Medical Subject Headings (MeSH) terms for Medline and Cochrane, and Emtree terms for EMBASE as appropriate for each term's tree. Their synonyms were included in the title, abstract and keyword searches. We performed the search on 26th June 2021 and updated the search on 18th May 2022. We provided the search terms for each electronic database (Supplementary Table 1). We hand-searched the bibliography of systematic reviews, as well as publications that we included, to ensure that our literature search was comprehensive.

Study selection
We uploaded the studies from the search strategy to Covidence, an internet-based software that facilitates collaboration between study team members during the study selection process. 19 SLC, MGSL, GOYG, JA, SYHW, and KCN performed screening in both phases. In the first phase, we screened the studies' titles and abstracts for relevance. In the second phase, we applied the eligibility criteria (as detailed above). At both stages, two reviewers independently screened the articles. Any conflicts were resolved by a third independent reviewer. The study team was not blinded to the study authors nor the institutions where the research was performed. For the metaanalysis, we included all studies that provided quantitative measures of the prior mentioned outcomes in both the intervention arm (PEWS) and the comparator arm (no PEWS). We excluded studies with an unacceptable risk of bias (refer Risk of Bias in individual studies) from the meta-analysis.

Data collection process and data items
Two independent reviewers extracted the data independently using a Microsoft Excel form, with variables determined a priori. SLC and MGSL piloted the data collection form first and refined the variables before SLC, MGSL, GOYG, JA, SYHW and KCN performed the extraction. We extracted the following: Study title, first author, year of publication, number of centres, country/countries involved, study design, and the study population. We also specified the PEWS used in the study, activation criteria, and the response arm (e.g. RRT or MET). For outcomes, we collected data on mortality, cardiopulmonary arrest, unplanned code events, and critical deterioration. Critical deterioration was a composite outcome that included significant clinical deterioration events (refer Eligibility criteria), 18 emergency or unplanned PICU admissions as defined by the authors. Where there was insufficient information, we aimed to contact the original study investigators for the required data, and follow-up in two weeks with another email for non-responders. If there was still no response after four weeks, we considered them un-contactable.

Risk of bias in individual studies
We used the Scottish Intercollegiate Guidelines Network (SIGN) to appraise the risk of bias in each study. 20 We assessed for risk of bias at both the study and the outcome level. Checklists were used depending on the study design (e.g. cohort or randomised controlled trial). Overall assessment ranged from unacceptable (0) to high quality (++). Two independent reviewers (GOYK and JA) assessed each article and differences were resolved by consensus.

Summary measures and synthesis of results
Extracted data were combined in a two-stage meta-analysis approach. In the first step, incidence rate rations (IRR) with their 95% confidence intervals (CI) were estimated from individual studies reporting patient-days such as number of deaths per 1000 patientdays. Likewise, risk ratios (RR) with 95% CI were estimated from individual studies reporting binary outcome data such as number of cardiac arrests in a hospital during a specific time period. In the second stage, a restricted Maximum Likelihood (REML) random effects meta-analysis was employed to combine RRs and IRRs from individual studies. Pooled results were reported as RR with 95% CI. Statistical heterogeneity was quantified using the I 2 statistic for analyses. 21 I 2 values >75% was considered as considerable amount of heterogeneity. We performed a sensitivity analysis by study design, pooling all studies at first regardless of study design and then only for the observational studies (thereby excluding the RCT). Meta-analysis results were presented in forest plot.

Risk of bias across studies
Publication bias was explored using visual inspection of the funnel plot, and Begg & Mazumdar's test was carried out to check the statistical symmetry of the funnel plot. 22

Certainty of evidence
We performed the certainty assessment using GRADE. This assessment was based on the study design, risk of bias, inconsistency, indirectness, imprecision and other considerations. 23 All analyses were conducted using Comprehensive Meta Analysis V5 and SAS 9.4 software.

Study selection and study characteristics
Among a total of 911 articles screened, 106 articles were assessed for eligibility and 15 were included for descriptive analysis. 12,18,32-36,24-31 ( Fig. 1) Articles were published between 2007 and 2018, mostly in the United States (7/15), followed by Australia (2/15) and Canada (2/15) ( Table 1). Fourteen of the 15 studies were pre-versus post-implementation studies with either prospective or retrospective controls, and one study was a multi-centre cluster randomised trial in multiple countries (Belgium, Canada, England, Ireland, Italy, New Zealand and Netherlands). 12 The other multicentre study of four sites in Canada compared postimplementation of PEWS to historical controls. 30 The majority of the studies were performed in the general paediatric population, with one in a paediatric cardiology unit and one in a paediatric oncology hospital (Table 1).
Studies used a wide range of PEWS and modified PEWS that spanned vital signs, clinical deterioration signs, biochemical abnormalities, and staff or parental concerns ( Table 2). While most studies involved an efferent arm consisting of either a RRT or MET in response to trigger criteria, others reported implementing staff training and modified escalation as part of their centres' PEWS-related interventions. In the only RCT included in this systematic review, the efferent arm remained status quo as per the existing system in each hospital. 12

Risk of bias within studies
We had methodological concerns regarding the comparability between the groups with PEWS versus no PEWS being investigated in pre-and post-implementation designs. Efforts to minimise bias and confounding were mostly absent, with some studies recognising potential confounding from concurrent quality improvement initiatives and co-interventions. 18,[25][26][27]29,30,32,33,35 There was no effort to blind the investigators who evaluated outcome data from the exposure (PEWS versus no PEWS), although there was recognition that knowledge of exposure status could have influenced the assessment of outcome. 27 The single RCT was not blinded. 12 The observational studies assumed patients in each arm had complete outcome data. Of the 15 studies, three studies were assigned an unacceptable risk of bias (RoB 0) and were excluded from the subsequent synthesis of results (Supplementary Table 2).
One of the 10 studies reported a significantly increased risk of mortality in the group without PEWS, compared to the group with PEWS (RR 1.52, 95% 1.33-1.74, p < 0.001). 27 This study utilised paediatric MET trigger criteria that were adapted from adult criteria with the addition of age-related abnormal readings (Table 2). Along with activation criteria, this study introduced a MET service after three months of extensive training for doctors and nurses.
Among six studies (413,370 hospital admissions) that reported the outcome of cardiopulmonary arrests, none of the studies demonstrated a significant difference with the introduction of PEWS. Overall, the risk of cardiopulmonary arrest without the implementation of PEWS was similar to the group with PEWS (pooled RR 1.22, 95% CI 0.93-1.59, p = 0.153, I 2 = 0.0%) (Fig. 4).
For the outcome of unplanned code events reported in four studies (168,544 hospital admissions), there was a significant increase in the risk of codes in the group without PEWS compared to the group with PEWS (pooled RR 1.73, 95%CI 1.01-2.96, p = 0.046, I 2 51.5%) (Supplementary Fig. 1). Two studies reported an increased risk without PEWS. One study reported an increased risk of hospital-wide code rates without PEWS (RR 3.56, 95% CI 1.43-8.84, p = 0.006) that decreased after introducing PEWS and RRT. 25 The other study Multi-centre cluster randomized trial General introduced activation criteria as part of a rapid response system, and found increased likelihood of codes without PEWS, but no increased risk of mortality or actual cardiopulmonary arrests. 30 Among six studies (156,031 hospital admissions) analysed for critical deterioration, there was no significant increased risk among those without PEWS compared to those with PEWS (pooled RR 1.21, 95% CI 0.90-1.62, p = 0.199, I 2 = 84.2%) (Fig. 5). The sensitivity analysis performed after the RCT was removed (11,492 hospital admissions) similarly yielded no significant relationship ( Supplementary Fig. 2). Two studies reported an increased risk of critical deterioration in the group without PEWS compared to the group with PEWS. The first was a study performed in a resourcelimited paediatric oncology hospital in Guatemala and reported increased risk of deterioration events resulting in unplanned PICU admissions for the group without PEWS (RR 1.43, 95% CI 1.13-1.82, p = 0.003). After the successful implementation of PEWS, the authors went on to find that PICU utilisation for inpatient transfers decreased significantly. 35 The second was a multi-centre cluster RCT across multiple countries that found a significantly higher risk of clinical deterioration in the group without PEWS compared to the group with PEWS (RR 1.67, 95% CI 1.34-2.08, p < 0.001). Each participating hospital kept their existing practice of RRT or MET.     8

Risk of bias across studies
We did not detect evidence of significant publication bias in the 4 outcome measures using funnel plots, and Begg and Mazumdar's test ( Supplementary Figs. 3-6).

Certainty of evidence
We found the certainty of evidence to be very low for all four outcomes chosen, given the serious risk of bias and imprecision for the cohort studies included in this systematic review and metaanalysis (Supplementary Table 3).

Discussion
We performed a systematic review and meta-analysis to compare clinical outcomes between patient populations with PEWS compared to those without. In populations without PEWS, we found an increased risk of mortality (pooled RR 1.18, 95% CI 1.01-1.38, p = 0.036) and unplanned code events (pooled RR 1.73, 95% CI 1.01-2.96, p = 0.046). The finding of increased mortality in the group without PEWS was not consistent when we performed a sensitivity analysis on cohort studies alone. Implementation of PEWS was not associated with a change in the rate or proportion of cardiopulmonary arrests (pooled RR 1.22, 95% CI 0.93-1.59, p = 0.153) or critical deterioration events (pooled RR 1.21, 95%CI 0.90-1.62, p = 0.199). An earlier systematic review published in 2017 looked at the effectiveness of PEWS for detecting clinical deterioration, effectiveness of PEWS response mechanisms and evidence for PEWS implementation strategies. 37 They found gains in early clinical interventions and potential improvements in patient safety through multidisciplinary coordination and teamwork but reported a lack of standardised outcomes to enable robust comparison between studies. 37 The authors also concluded that diversity in these scoring systems (with different weightage to physiological parameters, clinical findings and thresholds for action) made it difficult to compare the performance of PEWS between studies. We recognise that these differences in PEWS components and thresholds will continue to exist as various PEWS are derived from different patient populations, and between countries and centres with different human and technology resource availability. Therefore, we sought to perform a pragmatic systematic review to understand if the incorporation of PEWS with or without RRT/MET improved clinical outcomes irrespective of healthcare resource availability.
Most studies on PEWS have focused on validation of each respective tool, and a recent systematic review highlighted that some paediatric track and trigger tools (PTTT) have good diagnostic accuracy, primarily in the prediction of PICU transfers. 13 The same authors reported methodological concerns precluding recommendations on effectiveness of PEWS. They included studies that involved inpatients 0-18 years old, with outcome measures of mortality, critical events including unplanned admission to a higher level of care, cardiac arrest, respiratory arrest, medical emergencies requiring immediate assistance, acuity at PICU admission and PICU outcomes. In contrast, we chose to narrow our search to include only studies that actually implemented PEWS in their healthcare settings and had a comparator arm ("No PEWS"). We also limited the outcomes to those of mortality, cardiopulmonary arrest, unplanned code events and critical deterioration. These specifications allowed us to focus on the impact of PEWS within systems that had already imple-mented PEWS and allowed us to study commonly-used outcomes that improved comparability.
We recognise that the difference in outcomes reported among studies could firstly be attributed to variation in patient population.
Sharek et al reported a reduction in both mortality and code rates in a quaternary care children's hospital setting. 25 The authors recognised that their hospital serves children at high-risk for codes due to a high case mix, and demonstrated higher pre-intervention code rates compared to other centres. 25 Conversely, others with a relatively low rate of paediatric cardiac arrest postulate that the low incidence might itself lend to difficulty demonstrating a significant reduction of such events. 38 Secondly, various PEWS differ in composition of physiological criteria and trigger thresholds. 39 Many of these are either locally derived and internally validated, or modified from the published literature with external validation. 40 The concept of a universallygeneralisable PEWS is challenged by the fact that data-driven models reporting good accuracy are site-specific. 41 This means that each healthcare setting that seeks to implement PEWS must first and foremost have the capability for robust validation of these scores within their own populations. 40 Given the complexity of PEWS implementation, resource-limited settings may modify scores to ensure compliance and sustainability. 42 Some predictive scores are specialtyspecific scores, which may have better performance within those patient populations compared to PEWS derived from the generic paediatric population. 43 These variations could have accounted for the heterogeneity seen in our study.
We propose that the robustness and timeliness of the RRT/MET response is another important contributory factor to the heterogeneity seen. An accurate PEWS can facilitate early interventions provided there are successful implementation strategies in place. These should ensure that resuscitation occurs when the patients are clinically unstable but not yet pulseless, resulting in downstream gains. Tibballs et al demonstrated not only a reduction in preventable cardiac arrest, but also in survival from unexpected cardiac arrest on the wards, after the introduction of a MET service. 27 In another study, despite no significant reduction in mortality, the introduction of PEWS paired with a clinical response system resulted in a reduction in invasive ventilation among PICU admissions and shorter PICU stay. 33 In view of the low incidence of death and cardiopulmonary arrests, proximal outcomes like that of significant clinical deterioration events have been used to surrogate for late PICU admissions. Nevertheless, in the multi-centre RCT that found a significant decrease in significant clinical deterioration events, they did not find any impact on rates of cardiac arrest, urgent ICU admission, mortality after urgent ICU admission, risk-adjusted ICU mortality, or PICU resource use. 12 The authors recognised that late PICU admissions may constitute a minor proportion of overall PICU admissions and that such savings may not modify overall mortality or PICU resource use.
Aside from the outcomes chosen in our study, other benefits of PEWS implementation must be considered. These include improved communication between healthcare workers and changes to safety culture. 24,43 Adoption and maintenance of PEWS within the larger inter-disciplinary context provides opportunity to iteratively improve processes and upscale health technologies. 44,45 The implementation of a standardised system may also serve to eliminate variations in care, streamline interventions and provide opportunities to investigate "missed" cases of deterioration that are potentially actionable. 46 The PUMA (Paediatric early warning system Utilisation and Morbidity Avoidance) Programme is one such framework that pro-vides not only the PEWS tools but also guidance on implementation to support improvement initiatives. 47 We recognise that while we did not demonstrate a clear superiority among systems that implemented PEWS in reducing death, cardiopulmonary arrest or critical deterioration events, the absence of a demonstrable effect does not necessarily preclude the implementation of PEWS as part of a clinical response system that will enhance early recognition and intervention in the deteriorating child. Future research should focus on prospective evaluation of PEWS with clear documentation of the efferent arm interventions for patients at risk of decompensation from specific aetiologies including primary respiratory, circulatory or neurologic causes. 48

Limitations
We recognise that there are inherent limitations to the study designs employed by most of the studies. Regardless of prospective or historical controls, a pre-and post-implementation comparison is fraught by confounding, not least by the impact of improved quality of care with the passage of time. Therefore, it was not possible to establish causality between the implementation of PEWS and improved outcomes in many of these studies. Moreover, the introduction of PEWS would have resulted in hospital-wide education of physicians, nurses and allied health staff, and it was not possible to measure improvements in outcomes attributed to each of these factors. 27 Implementation of PEWS is complex, requiring adherence to protocols, effective communication and, overall, a motivated culture. We were unable to account for these in our systematic review. Finally, we acknowledge that not all cardiopulmonary arrests are the same, and that within this group some are more preventable than others. 26 By grouping all the cardiopulmonary arrest outcomes together, we potentially diluted the impact that PEWS implementation would have on these events.

Conclusion
We detected reductions in mortality and code rates among healthcare systems that implemented PEWS. We recognise that gains in clinical outcomes are dependent on healthcare setting, resource availability, and presence of robust efferent response systems.

Conflicts of Interest
There are no known conflicts of interest.

Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data sharing statement
Data are available on reasonable request, directed to the Corresponding author.
We would also like to thank Ms Peggy Fong Bih Yuh from KK Women's and Children's Hospital library for her assistance in the literature search.
CRediT authorship contribution statement