{'fraction_non_alphanumeric': 0.029861852534860397, 'fraction_numerical': 0.02601184121129768, 'mean_word_length': 4.976411446249033, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stillbirth rates, service outcomes and costs of implementing NHS England’s Saving Babies’ Lives care bundle in maternity units in England: A cohort study

Objective To assess implementation of the Saving Babies Lives (SBL) Care Bundle, a collection of practice recommendations in four key areas, to reduce stillbirth in England. Design A retrospective cohort study of 463,630 births in 19 NHS Trusts in England using routinely collected electronic data supplemented with case note audit (n = 1,658), and surveys of service users (n = 2,085) and health care professionals (n = 1,064). The primary outcome was stillbirth rate. Outcome rates two years before and after the nominal SBL implementation date were derived as a measure of change over the implementation period. Data were collected on secondary outcomes and process outcomes which reflected implementation of the SBL care bundle. Results The total stillbirth rate, declined from 4.2 to 3.4 per 1,000 births between the two time points (adjusted Relative Risk (aRR) 0.80, 95% Confidence Interval (95% CI) 0.70 to 0.91, P<0.001). There was a contemporaneous increase in induction of labour (aRR 1.20 (95%CI 1.18–1.21), p<0.001) and emergency Caesarean section (aRR 1.10 (95%CI 1.07–1.12), p<0.001). The number of ultrasound scans performed (aRR 1.25 (95%CI 1.21–1.28), p<0.001) and the proportion of small for gestational age infants detected (aRR 1.59 (95%CI 1.32–1.92), p<0.001) also increased. Organisations reporting higher levels of implementation had improvements in process measures in all elements of the care bundle. An economic analysis estimated the cost of implementing the care bundle at ~£140 per birth. However, neither the costs nor changes in outcomes could be definitively attributed to implementation of the SBL care bundle. Conclusions Implementation of the SBL care bundle increased over time in the majority of sites. Implementation was associated with improvements in process outcomes. The reduction in stillbirth rates in participating sites exceeded that reported nationally in the same timeframe. The intervention should be refined to identify women who are most likely to benefit and minimise unwarranted intervention. Trial registration The study was registered on (NCT03231007); www.clinicaltrials.gov.

evaluation, exploratory outcomes where these aid interpretation, and the results of the economic analysis.

Design and study population
We conducted a pragmatic analysis before and after the introduction of the SBL care bundle (the exposure of interest) using longitudinal data collected retrospectively in 19 NHS Trusts in England across 9 clinical networks. These trusts were purposively selected from a larger cohort of maternity units that had participated in the 2015 NHS Tracker Survey to provide a geographically distributed sample throughout England and a range of birth rates and levels of neonatal care and to reflect a range of levels of implementation at the time of selection. Units from London were not included as they were participating in a stepped-wedge cluster trial of implementation of the Growth Assessment Protocol [12]. These trusts were deemed early adopters as they were pilot sites for implementing SBL in 2015 ahead of the national launch in March 2016. For the purposes of this analysis April 2015 was defined as the start of implementation in these 19 Trusts and outcomes were assessed at the nominal dates of April 2013 ("before") and April 2017 ("after") respectively, providing a minimum of two years of data before and after implementation of the SBL care bundle.
Retrospective data collection for the period April 2013 to October 2017 was carried out between September 2017 and January 2018. Recruitment (for surveys) took place between August and December 2017, following approval by the Health Research Authority in June 2017 (Reference 17/WM/0197). The study was registered on www.clinicaltrials.gov (NCT03231007) in July 2017 before commencement of data collection, and the evaluation protocol was published before commencing data analysis [11]. Analysis was performed on anonymized data. The study was reported according to STROBE guidelines [13].

Variables and data sources
Definitions of data collection methods and outcomes are described in detail in the protocol [11]. In brief, outcome data was collected from routine electronic clinical data held in Trust data repositories where this was available, the data being provided or converted to monthly aggregate totals (e.g. number of births per month). Anonymised postcode data were used to derive the index of multiple deprivation (IMD) and a mean decile computed for each Trust; where this was not provided an estimate was derived as the mean IMD of a surrounding area based on population size and birth numbers.
This was supplemented by clinical audits (20 sequential women per time point and Trust) of growth and birth monitoring practice in unselected women (pre and post implementation), small for gestational age births (pre and post implementation) and women attending hospital with RFM (post-implementation only). Additionally, in the post intervention period women were asked to complete a survey postpartum prior to discharge (informed consent) which included questions on smoking behaviour and interventions received along with experiences of RFM (S1 File). All health professionals involved in delivering maternity care were invited to participate in an online or paper survey regarding their views and experiences of the SBL care bundle (S2 File) if they had been employed in their current Trust prior to the launch of the care bundle initiative in April 2015. Data on the degree of implementation of the SBL care bundle was self-reported by Trusts and implementation scores were derived by applying a score to a Likert scale whether each intervention in each element was implemented: a) all of the time (score 3), b) most of the time (score 2), c) half of the time (score 1), d) not much of the time (score 0), e) never (score 0), or f) not relevant (score 0). These were totalled to give an implementation score for each element. As there were different numbers of interventions within individual elements, the overall implementation score for each Trust was calculated as the mean of the implementation scores for the four elements, each expressed as a percentage of the maximum score.

Statistical analysis
Electronic data on stillbirths other clinical outcomes and smoking rates was provided as, or converted to, monthly counts of outcomes and appropriate denominators. Outcomes were fitted using quasi-binomial models to allow for the possibility of overdispersion, except for scan rates, which were fitted using quasi-Poisson models. Generalised linear models with logarithmic link functions were used to estimate a linear trend over time. Trust was included as a covariate thus time trends are estimated on a within-Trust basis. From this model we derived estimates of the outcome rates at dates 2 years either side of the nominal SBL start date of April 2015 and estimated the risk-(or rate) ratio between these two time points as a measure of the change over the implementation period. Models additionally containing a step change at the nominal or reported implementation dates were also considered, but proved to be uninformative as step-changes could not be detected and therefore are not presented.
In order to investigate the relationship with reported implementation status, generalised linear mixed models were fitted with Trust as a random intercept and assessment-date implementation level, Care Level (Tertiary v Secondary), IMD (mean decile) and month as fixed effects. A risk-ratio between no and full implementation is reported.
Audit data and women's questionnaire data were available for one or two time points and the outcomes are presented as proportions of women audited with exact Binomial 95% CI. Where there were data from 2 time points (pre and post implementation), risk ratios between the two time points were derived using binomial regression models with logarithmic link functions. Data analyses were conducted using R (www.R-project.org) [14].
The sample sizes for the various datasets were largely determined pragmatically by the constraints of time and the need to have a reasonable pre and post launch period to assess trends, with an inevitably staggered true implementation. Based on a prevalence of 4.7 normally formed singleton stillbirths per 1,000 total births in 2014 [4], the potential annual number of stillbirths detected was estimated as 470 per 100,000 total births across all study sites. Conservatively, a two year pre versus one year post-comparison was estimated to have 80% power to detect a drop in the primary stillbirth rate from 4.7 to 3.9/1000 -a 17% reduction [11].

Economic analysis
An economic analysis was carried out using the data generated from the comparisons outlined above alongside additional data reported by the Trusts on the resources they used to implement the SBL care bundle and the level of implementation they reported. These data were also used to estimate what annual costs and stillbirths avoided for the whole of England would be in the scenario that all Trusts implemented all elements of SBL care bundle.
The implementation cost consists of two parts: the direct cost of putting in place each element and the cost of the secondary effects (mode of delivery, induction of labour, ultrasound scans). The direct cost of implementing each element was estimated but it is not possible to determine which element the secondary effects relate to so they have been calculated for the SBL care bundle overall. No additional funding was provided to Trusts to implement the SBL therefore the direct implementation costs should be interpreted as the 'value' of the SBL rather than additional funding required.
Unit costs were derived from published sources [15,16], NHS partners [17], and through email communication with training/software providers. The fees for externally-provided training courses were included in the implementation cost. In-house training was assumed to form part of routine, ongoing continuing professional development (CPD), and so the cost of staff time to complete training was not included in the primary analysis, but was explored in a sensitivity analysis.
The total annual birth rate for England was identified from national published data [18]. The time-series-adjusted stillbirth rates from before and after the implementation date were applied to this number to estimate the difference in the number of stillbirths before and after implementation of the SBL care bundle i.e. the number of stillbirths potentially avoided.
It was necessary to make a number of assumptions in order to estimate the resources and costs associated with the care bundle. As such the costs and outcomes reported should be interpreted as 'best estimates'. The impact of varying some of the assumptions on costs and outcomes were explored in a series of one-way sensitivity analyses.

Results
The characteristics of the participating Trusts are described in Table 1. The annual birth rate ranged from 2,900 to 8,894. Thirteen out of the 19 Trusts (68%) were secondary care providers with the remainder being tertiary maternity units. The majority (n = 17) of Trusts provided care for women residing in areas between the 2 nd and 6 th deciles of deprivation. Implementation rates of the SBL care bundle ranged from 0 to 100%, with a mean value of 74%. Response rate on the patient questionnaire was between 9% and 60% of all births.

PLOS ONE
Trusts reported that the SBL care bundle was implemented in a phased and gradual manner, with most Trusts implementing some SBL care bundle elements prior to the SBL launch and 12/19 achieving >75% implementation by the end of the study period. Table 1 shows the implementation scores at the assessment date (April 2017) for each Trust with fuller details of the individual elements and time points available in S1 Fig. Many Trusts were unable to give precise dates for the implementation of individual elements of the SBL care bundle. Implementation rates varied between elements with only one Trust stating 100% for all four elements; intrapartum fetal monitoring was the most completely implemented element of the care bundle and screening for SGA infants the least.

Stillbirth rates
Stillbirth rates declined from 4.2 to 3.4 per 1,000 births over the 4 year period, which was statistically significant (P<0.001), RR of 0.80 (Table 2). Fig 1 shows a steady decline in the rate over time, and we were unable to demonstrate any step changes associated with implementation of the SBL care bundle. Term singleton and stillbirths associated with SGA showed similar decline ( Table 2). Pre-term stillbirth showed a slightly smaller decreased from 2.3 per 1,000 births to 1.9 per 1,000 births (Relative Risk 0.82, p = 0.014). We were unable to demonstrate

PLOS ONE
any association between stillbirth rates and the reported level of implementation of the SBL care bundle (Table 2). Table 2 summarises the clinical and service outcomes derived from Trust database systems. There were modest increases in the numbers of preterm births, induction of labour and emergency Caesarean sections (CS) over the evaluation period. The number of ultrasound scans per pregnancy increased markedly from 3.5/pregnancy to 4.3/pregnancy in the 8 Trusts providing complete data (S2 Fig). Other Trusts provided partial scan data which was consistent with these trends, but not sufficiently complete for formal analysis. An increase in the number of ultrasound scans performed was associated with a reduction in stillbirth rate (Adjusted Rate Ratio of -0.14 (SE 0.06) stillbirths per additional scan performed P = 0.026). Increases were also seen in elective CS and admissions to NICU, although it must be noted that the quality of data regarding NICU admission was poor. None of these outcomes showed a significant association with implementation of the SBL care bundle.

Process outcomes
The number of women smoking at delivery declined from 14.3% to 11.8% (P<0.001) over the evaluation period. However, this reflects a decline in the proportion of women smoking at booking rather than an increase in the numbers ceasing smoking during pregnancy (Table 3). 70.1% of women reported being offered a CO test and 99.1% of these women reported accepting the offer, however a smaller proportion of women who smoked were offered referral to smoking cessation services (60.1%) and only 31.0% attended this service. Trusts reporting full implementation of the element had higher proportion of women having a CO test at booking (RR 7.95, Table 3), but no difference in the proportion of women referred for smoking cessation services (RR 1.11, Table 3).
Over the time period of the evaluation, documentation of screening for SGA fetuses improved, with increased documentation of symphysis-fundal height and estimated fetal

PLOS ONE
weight on fetal growth charts. The documentation of these parameters was also significantly higher in Trusts with full implementation of this element (RR 5.07 (symphysis-fundal height), RR 38.24 (estimated fetal weight)). The documented increase in screening for SGA fetuses and in the frequency of ultrasound scan was associated with a 59% increase in the detection rate of SGA fetuses from 33.8% to 53.7% (Table 3). The leaflet regarding RFM was reported to be received by the majority of women (73.1%), and attendance with RFM occurred in 77.3% of episodes of RFM. A lower proportion of cases used a checklist to guide management (52.2%) and management varied from practice recommended in SBL care bundle with 74.4% of women with RFM having a fetal heart rate tracing and 29.4% of women receiving an ultrasound scan. A large proportion of women reporting RFM had an induction of labour (54.7%; Table 3). Women attending trusts with full implementation of the SBL care bundle were more likely to receive the information leaflet about RFM (RR 2.87, Table 3). Columns show the data source, the number of Trusts and women providing data, the fitted pre and � post implementation rates (±2y from the SBL launch date), the post v pre risk ratio (with 95% CI and significance level) and the risk ratio associated with 100% implementation of the relevant SBL element. � Adjusted for care level, IMD and calendar month. †Women-refers to data obtained via the postnatal questionnaire of women undertaken in the postnatal period. https://doi.org/10.1371/journal.pone.0250150.t003

PLOS ONE
We were unable to obtain sufficient data directly from participating Trusts regarding training in interpretation of fetal heart rate traces over the duration of implementation of the SBL care bundle. Data from the staff questionnaires showed that 56.5% of staff (n = 1,064) reported they had received training in the preceding year with 16.8% of those who had been trained reporting they had completed assessment in fetal heart rate interpretation. Staff employed the "buddy system" where at least two professionals interpreted the fetal heart rate trace in 50.2% of audited cases. Trusts with full implementation were more likely to have used the CTG "buddy system" and a standardised recording of the fetal heart rate assessment (RR 33.00).

Staff views
1,064 health professionals completed the staff survey (details of respondents can be found in S3 File); 78% of respondents were midwives. Overall, 58% (584/905) of respondents were aware of the SBL care bundle, with lowest rates of awareness in junior doctors and ultrasonographers (47% (7/15) and 32% (11/34) of respondents respectively). Almost all staff perceived an increased demand for ultrasound scans (97%, Table 4), rates of induction of labour (98%) and to a lesser extent Caesarean sections (80%). A significant proportion of respondents (41%) perceived that the stillbirth rate had decreased in their unit, although the same proportion perceived it was unchanged (Table 4). Most staff 82% (807/983) agreed with the statement that their unit was actively doing things to improve the safety of mothers and babies.

Economic analysis
There were 666,025 births (live births and stillbirths) in England in 2016 [18]. Based on data observed from the study sites, the number of women booked is on average 14% higher than the number of births, therefore, it was estimated that there were 759,299 bookings in the same period. The primary estimated cost of implementing the care bundle for the first year in England is £93,116,650, or approximately £140 per birth. The direct cost (excluding secondary costs) is £3,650,357 (4%) of the total cost. The full breakdown of costs are reported in Table 5.
A series of one-way sensitivity analyses were conducted to explore the impact of assumptions made to estimate costs. Selected sensitivity analyses are shown in Table 6 below (see S1 Table for all alternative assumptions explored). The assumption which reduced the estimated cost by the largest amount (-£14,123,076) was that only half of the increase in the number of inductions of labour observed were associated with the SBL care bundle (i.e. an increase of 10% rather than 20%). Including the cost of staff time to complete training courses associated with Elements 2 and 4 of the SBL care bundle increased the estimated cost by the largest amount (+£30,807,840).
As reported above, the stillbirth rate was 0.84/1,000 births lower in the participating Trusts after SBL care bundle was implemented than before. Applying this to the number of births in Table 4. Staff views about changes in practice and outcomes in their Trust over the time frame of adoption of the SBL care bundle (n = 1,064).

PLOS ONE
England in 2015/16 (n = 666,052), an estimated 559 stillbirths per year may have been avoided.
Using the primary estimate of the cost of the SBL care bundle (£93,116,650), this equates to £166,577 per stillbirth avoided. However, this figure should be interpreted with caution because it is not possible to determine the proportion of either the increase in secondary resource use or the reduction in stillbirths that are directly attributable to the SBL care bundle.

Discussion
This evaluation of the SBL care bundle found that implementation of the elements was variable, but increased from baseline levels in all early adopter sites. Over the same time frame stillbirth rates fell in the participating maternity units by 20% from 4.2 to 3.4 per 1,000 live births. This rate of reduction was greater than seen across the whole of England over the same period which reduced from 4.2 to 3.7 per 1,000 births from 2013-2017 (4.2 to 3.8 per 1,000 births in units not included in this evaluation) [19]. Data from MBRRACE from participating units suggests that the decrease in intrapartum stillbirths was particularly dramatic, falling from 0.32 per 1,000 births in 2013 to 0.06 per 1,000 births in 2016 (RR 0.18, 95% CI 0.08, 0.45) [3,20]. There were also important changes in secondary outcomes including a reduction in spontaneous vaginal births, increase in preterm births, Caesarean sections, ultrasound scans, and admissions to the neonatal unit. Due to the variation in implementation strategies, phasing and timing we were not able to identify a "step" in our time series analysis which would have suggested a direct association between implementation of the care bundle and the observed reduction in stillbirth or changes in secondary outcomes. However, the degree of implementation was related to important process measures including plotting of fetal growth and assessment of intrapartum fetal heart rate monitoring.

Strengths and limitations
This evaluation was strengthened by the adherence to a published protocol which used stillbirth as its primary outcome rather than a surrogate measure as often used in this area [11]. The evaluation was conducted in 19 different maternity units reflecting the diversity of UK maternity units and included approximately 14% of the births in the UK. The data presented here represent the major component of the Saving Babies' Lives Project Impact and Results Evaluation (SPiRE project) which also included analysis of clinical practice guidelines and service-user and staff views to provide a holistic view of the impact of implementing the SBL care bundle. The results of the other components are reported elsewhere [21]. This study is limited by it's before-and after study design which was employed pragmatically as the care bundle and its implementation were planned before the evaluation project was commissioned. The degree of implementation was self-reported by participating trusts which may not have accurately reflected the degree of implementation in the clinical service. Furthermore, the variation in levels, phasing, and strategies of implementation meant that there was no discrete moment within or across Trusts where a change in practice could be determined. The use of routinely collected data was both a strength and a limitation of the study. The use of routinely collected data allowed the project to achieve the necessary large sample size and to be undertaken in a comparatively short-time frame, but the use of these data were frequently challenging due to variation in codes and coding strategies employed by individual Trusts. Furthermore, routine data may include recognised coding errors in maternity health episode statistics [22], but as this would affect both the before and after time periods in this analysis the impact of this may be minimal. In some cases, this meant that data on certain items were excluded for some or all Trusts e.g. admission to neonatal unit and rate of hypoxic-ischaemic encephalopathy. This also meant that it was not possible to include costs (or potential cost-savings) associated with these items in the economic evaluation.

Meaning of the study: Possible explanations and implications for clinicians and policymakers
Reducing stillbirths (alongside reducing maternal deaths, neonatal deaths, and brain injuries) is an explicit priority of the UK government, with a target to reduce such events by 50% by 2030 and by 20% by 2020. Achieving this aim will require an annual rate of reduction for stillbirth of 3.3%, significantly greater than the 1.4% reported between 2000-15 [2]. If maintained, the annual rate of reduction described in the study sites (~5%) would be sufficient to achieve this ambitious target and represents the most rapid reduction in stillbirth rates in over 20 years and is on target to achieve the national ambition to halve the rate.
In common with earlier national care bundles in perinatal care (e.g. Necrotising Enterocolitis Care Bundle) this evaluation has demonstrated clear improvements in process outcomes, but it was more challenging to identify changes in element-specific clinical outcomes and relate these to the degree of implementation [23]. Although this evaluation was not able to determine whether implementation of the SBL care bundle or any individual element was associated with the observed reduction in stillbirth, the association is plausible. Detection of SGA infants, the strongest risk factor for stillbirth in the UK [24,25], improved by 59% and use of ultrasound scanning, the principal screening method for SGA infants increased by 24%. Importantly, increased frequency of ultrasound scanning was inversely associated with stillbirth, suggesting that a pathway of increased detection of SGA may reduce stillbirth. Unfortunately, causes of stillbirth, particularly fetal growth restriction, are poorly recorded in routinely collected data which prevents analysis to determine whether there has been a reduction in stillbirths from fetal growth restriction [26].

Unanswered questions and future research
Any argument proposing a link between implementation of SBL and the outcome measures must be weighed against the change in outcomes which were not associated with the SBL care bundle. For example, the 20% increase in elective Caesarean section, which is unlikely to relate to implementation of the SBL care bundle as it contains no recommendations about Caesarean section, is more likely to reflect changes in guidance from the National Institute of Health and Care Excellence published in 2011 [27]. In addition to changes in national policy, we were not able to identify specific initiatives which may have been implemented in participating Trusts contemporaneously. As the SBL care bundle continues to be rolled-out in UK maternity units further evaluation needs to be planned to identify potential causal effects on stillbirth and other clinical outcomes.
Stillbirths are relatively rare events in the UK therefore large, good-quality, datasets are critical to informing efforts to reduce the stillbirth rate. Health policy makers should consider the evaluation of healthcare programmes prior to their implementation, in order that the most robust evaluation possible can be incorporated to ascertain its effects on process and clinical outcomes. The AFFIRM study, which prospectively evaluated increased maternal awareness and staff education about RFM (i.e. Element 3 of the SBL care bundle) [28], demonstrated how a stepped-wedge cluster design can be used effectively in this context.
The analysis presented here provides information about the average effect of implementation of the care bundle across the 19 Trusts. However, we were not able to explore the reasons for the variation in levels of implementation between sites, thus further research is needed to explore the origins of variation seen in different settings. Analysis of clinical practice guidelines relating to the SBL care bundle in participating maternity units found wide variation in their quality [21]. An ethnographic study of a UK maternity unit with a strong safety record (which was not part of this evaluation) described six mechanisms in place to enhance safety including: clearly articulated, constantly reinforced standards of practice, behaviour and ethics and structural influences on mechanisms for safety (e.g. staffing levels and financial infrastructure) [29]. Thus, we hypothesise that influences accounting for variable implementation may include: human factors, leadership, availability of up to date, accurate clinical practice guidelines and sufficient resource to implement them. These merit exploration in future analyses and the information can then be used to inform future quality improvement programmes.
While the resources required to implement the SBL care bundle may seem large, they should be viewed from a national perspective. In 2014/15 approximately £2.5bn was spent in the NHS on maternity services (with 664,399 births) and in 2016/17 obstetric claims handled by the NHS Litigation Authority accounted for £4.3bn of new claims reported. Based on figures for 2014/15, the maternity tariff per birth was approximately £3,760; the direct costs associated with SBL care bundle are estimated to account for £5 of this with secondary costs estimated at £135 per birth. As the majority of Trusts participating in the study reported using GROW software to produce customised growth charts the costs to do so were included in the estimated costs, however it should be noted that this software was not specified in the SBL care bundle and that alternatives could be used. It was not possible to collect data regarding some important resources relevant to the implementation of SBL care bundle, for example the impact of incorporating additional elements into routine antenatal appointments without increasing the duration of appointments, or the costs associated with NICU admissions, and so it is acknowledged that there is uncertainty around the estimated costs. Future economic analysis of the impact of the SBL care bundle or similar initiatives should also consider the health and social care costs of stillbirth (estimated as £13.1M/year in 2018) [30], which are greater than for a live birth and also extend into pregnancies after stillbirth [31].

Conclusion
Prior to the launch of the SBL care bundle the UK stillbirth rate was 24 th out of 49 HICs and the annual rate of reduction was in the lowest third. Although adoption of the SBL care bundle was variable, all sites implemented elements to some degree. In the same time frame, process outcomes improved and stillbirth rates in the early adopter units fell faster than national rates. However, changes in care had important secondary effects particularly with regard to increased obstetric intervention and consequent resource requirements. This assessment has provided an important insight into the need for evaluation to be planned at the same time as the implementation of large-scale initiatives to ensure that the opportunity to collect data to inform the ongoing development of care is maximised.