{'fraction_non_alphanumeric': 0.03413613782284071, 'fraction_numerical': 0.03309026466401325, 'mean_word_length': 5.0495606326889275, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Development and validation of a simplified score to predict neonatal mortality risk among neonates weighing 2000 g or less (NMR-2000): an analysis using data from the UK and The Gambia

Summary Background 78% of neonatal deaths occur in sub-Saharan Africa and southern Asia, among which, more than 80% are in low birthweight babies. Existing neonatal mortality risk scores have primarily been developed for high-resource settings. The aim of this study was to develop and validate a score that is practicable for low-income and middle-income countries to predict in-hospital mortality among neonates born weighing 2000 g or less using datasets from the UK and The Gambia. Methods This analysis used retrospective data held in the UK National Neonatal Research Database from 187 neonatal units, and data from the Edward Francis Small Teaching Hospital (EFSTH), Banjul, The Gambia. In the UK dataset, neonates were excluded if birthweight was more than 2000 g; if the neonate was admitted aged more than 6 h or following discharge; if the neonate was stillborn; if the neonate died in delivery room; or if they were moribund on admission. The Gambian dataset included all neonates weighing less than 2000 g who were admitted between May 1, 2018, and Sept 30, 2019, who were screened for but not enrolled in the Early Kangaroo Mother Care Trial. 18 studies were reviewed to generate a list of 84 potential parameters. We derived a model to score in-hospital neonatal mortality risk using data from 55 029 admissions to a random sample of neonatal units in England and Wales from Jan 1, 2010, to Dec 31, 2016. All candidate variables were included in a complete multivariable model, which was progressively simplified using reverse stepwise selection. We validated the new score (NMR-2000) on 40 329 admissions to the remaining units between the same dates and 14 818 admissions to all units from Jan 1, to Dec 31, 2017. We also validated the score on 550 neonates admitted to the EFSTH in The Gambia. Findings 18 candidate variables were selected for inclusion in the modelling process. The final model included three parameters: birthweight, admission oxygen saturation, and highest level of respiratory support within 24 h of birth. NMR-2000 had very good discrimination and goodness-of-fit across the UK samples, with a c-index of 0·8859–0·8930 and a Brier score of 0·0232–0·0271. Among Gambian neonates, the model had a c-index of 0·8170 and a Brier score of 0·1688. Predictive ability of the simplified integer score was similar to the model using regression coefficients, with c-indices of 0·8903 in the UK full validation sample and 0·8082 in the Gambian validation sample. Interpretation NMR-2000 is a validated mortality risk score for hospitalised neonates weighing 2000 g or less in settings where pulse oximetry is available. The score is accurate and simplified for bedside use. NMR-2000 requires further validation using a larger dataset from low-income and middle-income countries but has the potential to improve individual and population-level neonatal care resource allocation. Funding Bill & Melinda Gates Foundation; Eunice Kennedy Shriver National Institute of Child Health & Human Development; Wellcome Trust; and Joint Global Health Trials scheme of Department of Health and Social Care, Department for International Development, Medical Research Council, and Wellcome Trust.


Introduction
An estimated 2·5 million neonatal deaths occurred in 2018, accounting for 47% of deaths among children younger than 5 years. 1 The burden of neonatal mortality is unequally distributed, with nearly 80% of these deaths occurring in sub-Saharan Africa and southern Asia. 1 Between 2000 and 2015, neonatal mortality declined more slowly than mortality among children aged 1-59 months. This disparity was particularly notable in sub-Saharan Africa, where the annual mortality reduction for newborn babies was less than half of that for 1-59 month-olds. 2 Slow progress in this region might be related to high incidences of preterm birth (<37 completed weeks of gestation) and low birthweight (≤2000 g), 3,4 poor access to care for neonates, 5,6 and health system capacity issues, including shortages of skilled providers, essential supplies, and basic equipment. [6][7][8] More than 80% of neonatal deaths in sub-Saharan Africa and southern Asia occur in babies with a low birthweight. 9 Low birthweight can result from being preterm, being small for their gestational age, or both. Mortality is twice as high in full-term neonates who are small for their gestational age than in full-term neonates who are of average size, and 15 times higher in preterm neonates who are small for their gestational age than in babies with either characteristic alone. 10 The lower the birthweight and gestational age, the higher the mortality risk. 9 Around 86% of neonates born at fewer than 28 weeks' gestation, and 41% of those born at 28-31 weeks, will die without access to intensive care; 11 more than 75% of neonates in sub-Saharan Africa and southern Asia have no access to such care. 7 Estimates suggest that neonatal special care, 5,7 including resuscitation, kangaroo mother care, feeding support or intravenous fluids, and management of respiratory distress, infections, and jaundice, could prevent 70% of preterm deaths and decrease prematurity-related causes of neonatal mortality by 58%. 12 Various systems for scoring illness severity and mortality risk in neonates have been developed, primarily for high-income settings (appendix p 6). Therapy-based approaches, such as the Neonatal Therapeutic Intervention Scoring System (NTISS), 13 categorise illness severity by the quantity and type of therapies administered. By contrast, the Score for Neonatal Acute Physiology (SNAP), 14,15 the Transport Risk Index of Physiologic Stability (TRIPS), 16 and other physiologybased approaches use objective, measurable parameters that vary with illness severity, such as blood pressure. Related models, such as the Clinical Risk Index for Babies (CRIB), 17,18 combine physiological parameters with perinatal factors, such as birthweight, to provide an overall mortality risk score. SNAP and CRIB are the most widely used systems and have been extensively validated. 19

Research in context
Evidence before this study 2·5 million neonatal deaths occur each year, among which 78% are in sub-Saharan Africa and southern Asia. More than 80% of these deaths occur in babies with a low birthweight who are small because they are preterm, small for their gestational age, or both. There has been slow progress in reducing neonatal mortality, which accounts for nearly half of deaths in children younger than 5 years, highlighting the need for scale-up of effective interventions for neonates who are at risk. We searched PubMed for studies published between Jan 1, 1992, and July 31, 2019, with the search terms "infant, newborn", and "infant mortality", or "infant, newborn, diseases, and mortality", or "infant, premature, diseases, and mortality", or "hospital mortality", and "severity of illness index", or "risk assessment", or "predictive value of tests", or "outcome assessment". Multiple risk scores for neonatal mortality, illness severity, and clinical instability have been developed for intensive care settings. Most of these risk scores are not feasible for low-income and middleincome countries, because they rely on laboratory-derived and therapy-derived parameters that are frequently unavailable, or on clinical observations that are not reliably measurable. A need remains for a highly predictive tool, feasible for use in resourceconstrained settings, to help providers objectively assess mortality risk in the most vulnerable babies.

Added value of this study
Using population-wide data from 110 176 neonates admitted to 187 hospitals across England and Wales, this study has derived and validated a mortality risk score for neonates weighing 2000 g or less (NMR-2000). To our knowledge, this is the largest dataset used to develop and validate a neonatal mortality risk score. NMR-2000 uses data on three parameters: birthweight, oxygen saturation (peripheral capillary oxygen) at admission, and highest level of respiratory support at any point within 24 h of birth. The model had very good discrimination and goodness-of-fit across the development and UK validation samples, with a c-index of 0·8859-0·8930 and a Brier score of 0·0232-0·0271. The simplified integer score, which can be measured and calculated at the bedside, showed predictive ability similar to the model using regression coefficients. In the Gambian dataset, which included 550 neonates at one hospital, the model had good discrimination and overall goodness-of-fit, with a c-index of 0·8170 and a Brier score of 0·1688. The simplified integer score showed similar performance, with a c-index of 0·8082. Complete data for scoring were available for 83% of neonates. These findings indicate that the NMR-2000 is valid for use in health facilities where pulse oximetry is available and underscore the fact that implementation in low-income and middle-income countries would require sensitisation regarding documentation of the three parameters used in the model.

Implications of all the available evidence
To reduce neonatal mortality worldwide, there is an urgent need to scale-up evidence-based interventions targeting the major causes of death. Our risk score could expedite recognition of severe illness and enable targeted delivery of care to small and vulnerable neonates, increasing effectiveness and efficiency of facility-based neonatal care in low-income and middle-income countries. Further research is required to validate NMR-2000 in low-resource settings using a larger sample, and to evaluate its usefulness for clinical decision making. The score has the potential to inform resource use, including nursing workload.

See Online for appendix
Notably, none of the aforementioned systems are practicable for routine use in low-income and middleincome countries (LMICs), because these systems rely on laboratory-derived and therapy-derived measures that are often not available, or on clinical observations that are not reliably measurable in these settings. [20][21][22] The simplified age-weight-sex (SAWS) score is the only validated neonatal mortality score designed for low-resource settings. Among a derivation cohort of 428 neonates weighing 1500 g or less in Bangladesh and Egypt, the SAWS was reported to have moderate discrimination for in-hospital mortality. 22 To improve the quality of facility-based neonatal care in LMICs, a highly predictive tool, which is feasible for routine use, is needed to help providers objectively assess mortality risk in small babies.
This study has two parts: (1) model development using data from the UK, and (2) model validation using data from the UK and The Gambia. The objectives were to evaluate existing neonatal illness severity and mortality risk scores to select candidate variables for use in the new model; develop and validate a score feasible for use in LMICs to predict in-hospital neonatal mortality risk among neonates weighing 2000 g or less within 24 h of birth; and compare the performance of the novel score (NMR-2000) with that of an existing score (CRIB-II).

Study design and participants
This retrospective study used data held in the UK National Neonatal Research Database (NNRD) from 187 neonatal units to develop a model for scoring inhospital neonatal mortality risk in LMICs. The NNRD holds de-identified patient-level data, recorded by healthcare providers as part of routine care, from admissions to National Health Service neonatal units in England starting from 2008, and in Wales and Scotland starting from 2012. This study included neonates admitted to units in England and Wales between Jan 1, 2010, and Dec 31, 2017 (appendix p 1). The following exclusion criteria were also applied: birthweight more than 2000 g; being admitted at older than 6 h or following discharge home; neonates who were stillborn; neonates who died in the delivery room; neonates who were moribund (received only comfort care before death; appendix p 1).
As well as data from the NNRD, we used data on neonates in The Gambia. West and central Africa have the highest neonatal mortality worldwide (31 in 1000 livebirths). 1 In The Gambia in 2018, the neonatal mortality (26 in 1000 livebirths) ranked ninth among the 16 countries of west Africa. 1 An estimated 12% of Gambian neonates are born preterm. 3 Edward Francis Small Teaching Hospital (EFSTH) in Banjul is the national referral hospital where the neo natal unit admits around 1400 neonates annually. From 2010 to 2013, case-fatality was 35% overall and prematurity-related complications were the leading cause of death. 23 The Gambian cohort included all neonates weighing less than 2000 g who were admitted to EFSTH between May 1, 2018, and Sept 30, 2019, who were screened for but not enrolled in the Early KMC (eKMC) trial (NCT03555981). Some routine data, including mode of delivery and treatments administered, were collected from medical charts by trained study personnel. Other data collected as part of the screening process were exported from the trial database, including birthweight, sex, birth plurality, referral status, and peripheral capillary oxygen saturation (SpO 2 ; appendix p 1).
Model development and validation using the UK dataset was approved by the North West-Preston Research Ethics Committee (17/NW/0709), the UK Health Research Authority, and the London School of Hygiene and Tropical Medicine (LSHTM; reference number 14594). Letters were sent to the UK Neonatal Collaborative Lead of all units contributing data to the NNRD, providing information about the study and giving each an opportunity to opt out. Model validation using the Gambian dataset was approved by research ethics committees of the Gambian Government and Medical Research Council Unit The Gambia at the LSHTM (reference number 1643) and LSHTM (reference number 16189). Consent was not obtained, as this was a retrospective study using de-identified data.

Selection of candidate variables
To select the candidate variables for the model, 18 studies describing existing systems for assessing neonatal mortality risk and illness severity were reviewed to generate a list of potential parameters (appendix p 6). Parameters that are typically unavailable, infrequently obtained, or unreliably measured in low-resource settings were excluded (appendix p 1). Remaining parameters were evaluated using the following exclusion criteria: low prevalence in the NNRD (<0·1%); high proportion of missing data in the development dataset (≥20%); not predictive of mortality in neonates who are preterm or have a low birthweight; low prevalence within the first 24 h of life; little evidence to support validity; and concept better represented by an alternative variable (appendix pp 7-9).

Model development
To create the model, we used a development sample of neonates admitted to a random sample of neonatal units in England and Wales from Jan 1, 2010, to Dec 31, 2016. Logistic regression models were derived to model inhospital mortality risk. Robust standard errors allowed for clustering within units. All candidate variables were included in a complete multivariable model, which was progressively simplified using reverse stepwise selection, with the least statistically significant variable removed at each step. Discrimination was assessed with the c-index, equivalent to the area under the receiver operating characteristic (ROC) curve. A value of 0·5 indicates no predictive ability, 0·8 is considered good, and 1 is perfect. 21 Overall goodness-of-fit was assessed with the Brier score and calibration using plots of observed versus predicted risk (appendix p 2). Multiple imputation with chained equations was used to assess the effect of missing data (appendix p 2). The logistic regression model was executed across the imputed datasets, and the resulting β coefficients and c-index were compared with original estimates. A sensitivity analysis excluding neonates whose admission age was unknown (anonymised data derived from calculated difference between birth time and admission time) was done to reassess model performance, because admission at more than 6 h of age was an exclusion criterion. Performance was additionally reassessed follow ing exclusion of neonates who were transferred for any reason because outcome data were not available for these babies. Performance for predicting mortality within 24 h of birth was evaluated in a secondary analysis, because 36% of neonatal deaths occur within this timeframe. 24

Score development
To develop the score, we assigned the parameters in the final model points proportional to their β regression coefficient values. Whole numbers were used to generate an easily calculable score. We arbitrarily defined low-risk, medium-risk, and high-risk groups (appendix pp 2-3). To assess the calibration of the score to the model using regression coefficients, observed risks in groups and population deciles of scores were derived and compared with mean predicted risks in each group or population decile. We assessed overall predictive ability of the score using the c-index.

Model validation
We then evaluated both the internal and external validity of the model. Internal validity is the reproducibility of a prediction model for the underlying population from which the data originated. 25  =peripheral capillary oxygen saturation. FiO 2 =fraction of inspired oxygen. *Defined as an increase that was clinically significant enough to necessitate obtaining a culture to evaluate for suspected sepsis, at any point within 24 h of birth. †Defined as delivery of supplemental oxygen (FiO 2 >0·21) via any method at any point within 24 h of birth. ‡Not including initial resuscitation at birth; level 1 defined as nasal cannula or headbox; level 2 defined as continuous positive airway pressure, bilevel or synchronised intermittent positive airway pressure, or invasive ventilation with an endotracheal tube or tracheostomy. §Defined as birthweight less than the 5th percentile for gestational age, using UK-WHO standards. 26  sample; estimates of optimism were averaged and subtracted to provide optimism-adjusted measures. External validity is the generalisability of a model's performance to related populations. 25 The model was evaluated in three external validation samples: the random sample, which included neonates admitted to the units withheld from the development sample; the temporal sample, which included neonates admitted to units in England and Wales from Jan 1, to Dec 31, 2017; and the Gambian sample, which included neonates admitted to EFSTH between May 1, 2018, and Sept 30, 2019. Each sample was used to assess distinctive features of model performance. The random sample tested performance in different care settings in the UK within the same timeframe, whereas the temporal sample tested performance during a later timeframe. The Gambian sample was used to test performance in a LMIC care setting. We assessed model performance in each validation sample separately and in the UK full (combined random and temporal samples) validation sample. Discrimination was evaluated using the c-index, and goodness-of-fit was evaluated using the Brier score. Calibration was assessed by plotting observed versus predicted risk. We assessed the overall predictive ability of the risk score using the c-index.  In the Gambian sample, we redefined low-risk, mediumrisk, and high-risk groups to account for increased case fatality in this sample compared with the UK samples (appendix p 3). Observed risks in groups and population deciles of scores were derived and compared with mean predicted risks in each group or population decile of the Gambian sample.

Comparison with the CRIB II score
The NNRD did not include all the variables required for calculation of CRIB, SNAP, SNAP-II, SNAPPE-II, TRIPS, or TRIPS-II scores (appendix pp 7-9); therefore, CRIB-II was selected for comparison with NMR-2000 (appendix p 3). Because CRIB-II has only been validated for use in neonates born up to 32 weeks' gestation, 18 we compared c-indices for CRIB-II and NMR-2000 among neonates born at 32 weeks' gestation or earlier in the full validation sample. All analyses were completed using Stata (version 15).

Role of the funding source
The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results
For the selection of candidate variables, 18 studies were reviewed to generate a list of 84 potential parameters. 45 (53·6%) of 84 parameters were considered infeasible for LMICs and were excluded, among which, 25 (55·5%) also had a low prevalence or were not included in the NNRD (figure 1). Eight (9·5%) of 84 parameters were excluded because the evidence was scarce, because the parameters had poor predictive ability in preterm neonates or neonates with low birthweight, or because the parameter had a low prevalence within 24 h of birth. 18 candidate variables were selected for inclusion in the modelling process (panel). 110 176 neonates were included in the UK development and validation samples. Characteristics of the samples and participants are shown in table 1. More than half (56·6-58·2%) of the neonates had low birthweight (1500-2000 g), 28·0-28·3% had very low birthweight (1000-1499 g), and 13·7-15·1% had extremely low birthweight (<1000 g). Around half (50·4-51·3%) of the neonates were moderate-late preterm (32-36 weeks) and one-third (32·4-33·5%) were very preterm (28-31 weeks). Overall case-fatality was similar across samples (2·8-3·2%). Case-fatality of neonates with extremely low birthweight in the temporal sample (280 [12·5%] of 2238) was lower than in the other samples. No neonatal units declined to contribute data.
The full model (18 variables) had a c-index of 0·9223 in the development sample (n=41 514). After stepwise elimination, the final model included three variables (table 3), with a c-index of 0·8883 and a Brier score of 0·0232 (table 4). Complete data on all three variables were available for 46 108 (83·8%) of 55 029 neonates in the development sample. After imputation of missing values for predictor variables (n=54 956), the resulting β coefficients were nearly identical to original estimates (appendix p 9) and model performance was unchanged (c-index 0·8894; appendix p 2). Admission age was uncertain for 5 (0·01%) of 55 029 neonates; in a sensitivity analysis excluding these neonates, there was no change in performance (c-index 0·8886, Brier score 0·0232).  12 793 (23·3%) of 54 956 neonates were transferred from the unit of admission to another care unit; an analysis excluding these neonates showed improved performance (c-index 0·9150, Brier score 0·0255; appendix p 2). Predictive accuracy for mortality within 24 h (c-index 0·8858, Brier score 0·0037) was nearly identical to that for in-hospital mortality. Because availability of SpO 2 monitoring is variable in LMIC settings, we tested a related variable (clinically relevant increase in apnoea or bradycardia, oxygen requirement, ventilatory support, or respiratory rate); however, this variable was not associated with in-hospital mortality (c-index 0·5061). A plot of observed versus predicted mortality risk in the development sample is shown in figure 2. Birthweight was the most predictive variable in the model (c-index 0·8540). The reciprocal of the coefficient for birthweight divided by 100 ([1/-0·0032]/100=-3·13) was used as the constant to enable retention of exact birthweights in score calculation (table 3), 28 thereby improving predictive ability. The score range for low risk was set at 16 or more, for medium risk at 6-15, and for high risk at 5 or fewer points (appendix p 9). An example score form is shown in figure 3. Among 46 108 neonates from the development sample with complete data on the three variables included in the final model, 27 289 (59·1%) were designated as low risk, 17 215 (37·3%) as medium risk, and 1640 (3·6%) as high risk. Observed risks were 0·3% (95% CI 0·3-0·4) for low risk, 4·1% (3·8-4·4) for medium risk, and 27·3% (25·2-29·5) for high risk, with a c-index of 0·8875. Mean predicted risks derived from regression coefficients were 0·2% (SD 0·2) for low risk, 4·6% (SD 4·2) for medium risk, and 23·5% (SD 8·8) for high risk. Observed risks across population deciles by score were similar to the risks predicted with regression coefficients (appendix p 10).

Discussion
This population-wide study, including data from 110 176 newborn babies at 187 hospitals in the UK and 550 newborn babies at one hospital in The Gambia, has derived and validated NMR-2000 for predicting in-hospital mortality. A strength of this work is that, to our knowledge, this is the largest dataset that has been used to develop and validate a neonatal mortality risk score. Among neonates born at 32 weeks' gestation or earlier, the discriminatory ability of NMR-2000 was superior to that of CRIB-II, one of the most widely used neonatal risk scores.
Performance of the NMR-2000 simplified integer score, which can be measured and calculated at the bedside, was similar to that of the model using regression coefficients. The three parameters used in the score can be feasibly collected in LMIC settings. Although sub-Saharan Africa and southern Asia account for 78% of the world's neonatal deaths, 1 existing risk scores have primarily been developed for intensive care settings and often require complex calculations. In LMICs, where parameters in widely used scores are typically not available nor reliably measurable, 20,21 NMR-2000 could support shared decision making by enabling providers to objectively assess illness severity. 29 The score could be used in clinical trials to assess eligibility and compare participants. 29 Additionally, NMR-2000 could inform service delivery planning by identifying bottlenecks in care provision. 6 Given that 73% of neonatal deaths occur within the first 7 days of life, 24 early recognition of severe illness and rapid initiation of evidence-based interventions are crucial to promoting survival. 5,9,12 The c-index was 0·8859-0·8930 across the development and UK validation samples, suggesting that NMR-2000 can discriminate neonates who will die from neonates who will survive. This level of performance is similar to that of commonly used neonatal mortality scores in highresource settings. Discriminatory ability at the time of model derivation ranged from 0·87 for TRIPS-II 16 to 0·92 for CRIB-II. 18  The performance of NMR-2000 (c-index 0·8523) was superior to that of CRIB-II (0·7443) among neonates born at 32 weeks' gestation or earlier. The model also showed very good predictive ability for mortality within 24 h of birth (c-index 0·8858), which is notable because 37% of neonatal deaths in sub-Saharan Africa occur within this timeframe. 24 The NMR-2000 model showed a high level of agreement between observed and pred icted deaths, as assessed by calibration plots, in the develop ment and validation samples. Calibration plots are the preferred method for assessing calibration. 25 Previous neonatal scores, including NTISS, 13 SNAPPE-II, 15 CRIB-II, 18   TRIPS-II, 16 were reported to have good calibration for predicting in-hospital mortality using the Hosmer-Lemeshow test. However, such results should be interpreted with caution given the limitations of this test, which include subjective and imprecise grouping of babies as well as inability to denote the directionality of miscalibration when incongruities are detected. 25 In the Gambian validation sample, the NMR-2000 model had good discrimination and overall goodness-of-fit, with c-index of 0·8170 and Brier score of 0·1688. Complete data were available for 83% of neonates. The calibration plot showed a strong agreement between observed and predicted mortality. These findings suggest that NMR-2000 is valid for use in LMIC settings where pulse oximetry is available. Discrimination of the SAWS score, developed for neo nates weighing 1500 g or less in low-resource settings, at the time of validation (c-index 0·679-0·698) 22 was decreased relative to NMR-2000 among Gambian neonates weighing 1500 g or less (c-index 0·8069). Notably, neither goodness-of-fit nor calibration were reported for the SAWS score. 22 Further, SAWS relies on accurate assessment of gestational age, which can be challenging in LMICs because of late presentation for antenatal care, poor recall of last menstrual period, and unavailability of ultrasonography. 30 Case-fatality of Gambian neonates in this study is similar to that reported from a previous study at EFSTH (35% overall, 58% for neonates with a very low birthweight), 23 and higher than studies at similar hospitals in Ghana (20% overall), 31 Nigeria (14-20% overall), 32,33 and Burkina Faso (15% overall). 34 Among the three NMR-2000 parameters, all except SpO 2 are included on routine admission forms at EFSTH, Gambia (at time of screening), 23 as well as standard forms at government hospitals in Kenya, Malawi, and Tanzania. We were able to obtain SpO 2 data for the Gambian sample primarily because these data were being collected as part of the eKMC trial screening process. Variability in the implementation of routine pulse oximetry is a crucial gap in low-resource neonatal units. 35,36 In a study of nearly 7500 neonates admitted to 11 hospitals in Nigeria, hypoxaemia increased the adjusted odds of mortality by six times, and clinical signs (eg, chest in-drawing, grunting) poorly predicted hypoxaemia. 37 Furthermore, expansion of neonatal inpatient care, often of variable quality and frequently inclusive of unmonitored 100% oxygen supplementation, has placed sub-Saharan Africa on the brink of an epidemic of retinopathy of prematurity. 38 Widespread availability of SpO 2 monitoring and improved coverage of screening for and treatment of retinopathy of prematurity will be essential to control the incidence of visual loss in affected neonates. In LMIC settings, successful implementation of NMR-2000 would require sensitisation around recording the three parameters. Several studies have highlighted issues surrounding the collection of data on neonatal care in LMICs, including variable uptake of standard admission forms; 36 incomplete documentation of assessments, monitoring, and therapies prescribed; 20,23,36 and low capacity of data systems to capture information on neonates who die soon after birth or are transferred to another facility. 4 Increasing the quality and coverage of data is crucial to promote actions to improve neonatal survival, and will require coordination across different levels of the health-care system.
One strength of this study is our use of a large and purposely-selected UK dataset, which enabled maximisation of model performance. One limitation is that the Gambian dataset was small and limited to a single hospital; research is required to validate the model using a larger LMIC dataset. Several candidate variables in the development sample had a considerable pro portion of missing data, including admission heart rate, respiratory rate, and SpO 2 . It was not possible to compare NMR-2000 with the CRIB, SNAP, SNAP-II, SNAPPE-II, or TRIPS-II scores, because the NNRD did not include all parameters required for their calculation. Because pulse oximetry is not always available in low-resource neonatal units, the usefulness of the NMR-2000 score in such settings could be limited. The NNRD did not include clinical signs of respiratory distress that could be tested as a potential proxy for SpO 2 . We tested a related variable (clinically relevant increase in apnoea or bradycardia, oxygen requirement, ventilatory support, or respiratory rate); however, this variable was not associated with mortality. The use of respiratory support level as a parameter could affect the performance of the model. Administration of therapies varies in line with variations in clinical practice 19 and resource availability and so might not reflect true therapeutic requirements. 36 In LMICs, delivery systems for oxygen therapy and CPAP might be unavailable or non-functional, and related supplies (eg, nasal cannulas) might be out of stock. 8  Research is required to validate the NMR-2000 score in low-resource settings using a sufficiently sized dataset, and to evaluate its usefulness for supporting clinical decision making. 29 A follow-up study using a large, multihospital dataset from Kenya is planned. Nurses have essential roles as frontline providers of neonatal care; however, there is a severe shortage of neonatal nurses in LMICs. 6,7 Future research could explore the model's ability to inform resource use, 13 particularly nursing workload.
The NMR-2000 is a simplified risk score, validated for high-resource and low-resource settings where pulse oximetry is available, to accurately predict in-hospital mortality among neonates weighing 2000 g or less. By enabling providers to objectively assess illness severity, this tool could contribute to improvements in the quality of care delivered in LMIC facilities. Early recognition of severe illness and rapid initiation of evidence-based interventions are crucial to promoting survival of small and vulnerable neonates.

Declaration of interests
CG has received grants from the Medical Research Council during the conduct of the study; grants from the National Institute for Health Research, Mason Medical Research Foundation, Chiesi Pharmaceuticals, Rosetrees Foundation, and the Canadian Institute for Health Research, outside the submitted work; and personal fees from Chiesi Pharmaceuticals, outside the submitted work. CG is also a voluntary, unremunerated member of the Neonatal Data Analysis Unit Steering Board, which oversees the UK National Neonatal Research Database. JEL is a member of the International Advisory Board for The Lancet Child and Adolescent Health. All other authors declare no competing interests.

Data sharing
Data from the UK National Neonatal Research Database is available upon request.