{'fraction_non_alphanumeric': 0.039074376442767865, 'fraction_numerical': 0.029390237036202915, 'mean_word_length': 4.746360401164671, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparison of the Performance Between Sepsis-1 and Sepsis-3 in ICUs in China: A Retrospective Multicenter Study

ABSTRACT The definition of sepsis was updated to sepsis-3 in February 2016. However, the performance of the previous and new definition of sepsis remains unclear in China. This was a retrospective multicenter study in six intensive care unit (ICUs) from five university-affiliated hospitals to compare the performance between sepsis-1 and sepsis-3 in China. From May 1, 2016 to June 1, 2016, 496 patients were enrolled consecutively. Data were extracted from the electronic clinical records. We evaluated the performance of sepsis-1 and sepsis-3 by measuring the area under the receiver operating characteristic curves (AUROC) to predict 28-day mortality rates. Of 496 enrolled patients, 186 (37.5%) were diagnosed with sepsis according to sepsis-1, while 175 (35.3%) fulfilled the criteria of sepsis-3. The AUROC of systemic inflammatory response syndrome (SIRS) is significantly smaller than that of sequential organ failure assessment (SOFA) (0.55 [95% confidence interval, 0.46–0.64] vs. 0.69 (95% confidence interval, 0.61–0.77], P = 0.008) to predict 28-day mortality rates of infected patients. Moreover, 5.9% infected patients (11 patients) were diagnosed as sepsis according to sepsis-1 but not to sepsis-3. The APACHE II, SOFA scores, and mortality rate of the 11 patients were significantly lower than of patients whose sepsis was defined by both the previous and new criteria (8.6±3.5 vs. 16.3±6.2, P =  < 0.001; 1 (0–1) vs. 6 (4–8), P = <0.001; 0.0 vs. 33.1%, P = 0.019). In addition, the APACHE II, length of stay in ICU, and 28-day mortality rate of septic patients rose gradually corresponding with the raise in SOFA score (but not the SIRS score). Sepsis-3 performed better than sepsis-1 in the study samples in ICUs in China.


INTRODUCTION
Although sepsis has attracted the attention of physicians and researchers because it is the leading cause of mortality and a major health burden worldwide, the definition of sepsis remains debatable (1)(2)(3)(4)(5)(6). Therefore, in February 2016, based on clarity and content validity, and following literature reviews and expert deliberation, the Third International Consensus Task Force convened by the European Society of Intensive Care Medicine and the Society of Critical Care Medicine (the Consensus) redefined sepsis as ''life-threatening organ dysfunction due to a dysregulated host response to infection'' (7,8). It then revised and validated the new clinical criteria for sepsis (sepsis-3) using US electronic healthcare databases (9).
Intensive care units (ICUs) are rare recourses worldwide. China is a mid-income developing country with limited healthcare resources, so a more precise strategy is needed for admitting very severely ill patients to ICUs (10)(11)(12). Sepsis-1, which defined sepsis as a host's systemic inflammatory response syndrome (SIRS) to infection, and the presence of symptoms meeting two or more SIRS criteria as an inflammatory response, was always criticized because of its poor specificity (5,6,(13)(14)(15)(16). In 2001, the Consensus recognized this limitation but failed to offer alternatives because of the lack of supporting evidence (17). Until now, the main diagnostic criteria for sepsis used in ICUs in China are still sepsis-1 (infection plus two or more SIRS). Validation of the more specific sepsis-3 criteria is therefore urgently needed in China.
We conducted the present retrospective multicenter study to evaluate the performance of sepsis-1 and sepsis-3 in China using the area under receiver operating characteristic (AUROC) curves to predict 28-day mortality rates. We also investigated the possibility of using sequential organ failure assessment (SOFA) scores to grade the severity of sepsis.

Ethical statement
The Medical Ethical Committee of Zhejiang University approved the study. The need for informed consent was waived because of the retrospective and observational design of the study.

Settings
The present study was conducted at six intensive care units with approximately 100 beds in total in five Zhejiang University teaching hospitals (approximately 10,000 hospital beds). All the septic patients were treated according to the International guidelines for the management of severe sepsis and septic shock (2012) (18).

Patients
In total, 769 critically ill patients were consecutively admitted to the study ICUs between May 1, 2016 and June 1, 2016. Patients were excluded from the study for the following reasons: younger than 18 years old, ICUs stays of less than 24 h, history of organ transplantation, for those patients are cared by another group of specialists, pregnancy, information missing but needed to fulfill the sepsis-1 or sepsis-3 criteria. If patients were admitted to the same ICU more than once for the same episode of sepsis, only the first ICU admission was counted. The final number of patients enrolled in the study was 496. Figure 1 shows details of inclusion and exclusion criteria for the study.

Diagnosis
Infection was suspected in all patients who had cultures submitted and who were started on antibiotics within a 24 h window.
Sepsis-1 defines sepsis as infection-induced SIRS (13). The clinical criterion is suspected infection plus SIRS. A diagnosis of SIRS is appropriate where at least two of the following signs are present: heart rate faster than 90 beats per min; respiratory rate faster than 20 breaths per min; leukocyte count greater than 12,000/mL or less than 4,000/mL; temperature less than 368C or greater than 388C. The present study assessed the SIRS criteria within the first 24 h of admission to ICU.
Sepsis-3 defines sepsis as a life-threatening organ dysfunction due to a dysregulated host response to infection (7)(8)(9). The clinical criterion is suspected infection plus SOFA !2. In the present study, SOFA scores were recorded within the first 24 h of admission to ICU. The different criteria of sepsis-1 and sepsis-3 to candidate sepsis in patients with suspected infection are listed in Table 1.

Data collection
Two trained research assistants retrospectively collected the following data from the electronic clinical records: gender, age, admission origin (from emergency room or not), relevant information about infection (clinical, imaging, and microbiological signs), procalcitonin (PCT), and C-reactive protein levels, length of ICU and hospital stays, and 28-day mortality outcome (survival or death). We recorded the whole medical cost from the electronic clinical records of the hospitals, and converted them to US dollar according to the current exchange rate on June 26, 2016 when we analyzed the data. We also extracted information for SIRS, Acute Physiology and Chronic Health Evaluation (APACHE) II and SOFA scores within the first 24 h of admission to ICU.
The data were stored in a Microsoft Access database. Another two trained research assistants randomly extracted 10% of cases from the database to check the completeness, accuracy, and relevance of the information.

Statistical analysis
Variables with a normal distribution were presented as mean AE standard deviation (SD); variables with a skewed distribution were presented as medians and quartiles. Numerical data were presented as numbers and percentages, as appropriate. Groups were compared with t tests or non-parametric Kruskal-Wallis tests, where applicable. The chi-square test was used for comparison of frequencies. The Jonckheere-Terpstra test was used for comparison of continuous variables. AUROC of SIRS and SOFA scores were plotted and used to predict 28-day mortality rates of the groups. Binary multivariable logistic regression (forward LR) was used to determine the contribution of the SOFA score to the sepsis outcome. All tests were two-sided with a significance level of P < 0.05. All statistical analysis was conducted using the Statistical Package for the Social Sciences (IBM SPSS 20.0).

Demographics
Of 769 patients admitted to the six ICUs during the study period, 496 patients were enrolled in the study. The average age of the 496 patients was 58.3 AE 17.1 years old; 59.1% were male, and the mean APACHE II score was 13.5 AE 6.4. The total 28day mortality rate was 20.4%, and patients' ICU and hospital stays were 6 (4-12) and 21 (11-34) days, respectively. The enrollment process is summarized in Figure 1.
Patients diagnosed with sepsis-3 were significantly older than the non-sepsis patients; however, there was no difference in age between sepsis patients diagnosed with sepsis-1 and non-sepsis patients. The APACHE II and SOFA scores in the first 24 h after admission to ICU were significantly higher in sepsis patients as defined by both sepsis-1 and sepsis-3. Sepsis patients diagnosed using sepsis-1 or sepsis-3 stayed longer in ICU than non-sepsis patients. The mortality of sepsis-1 patients was 31.2% (58/186), and of sepsis-3 patients 33.1% (58/175). Detailed characteristics are shown in Table 2.

SIRS in infected patients
All suspected infection patients fulfilled two or more SIRS criteria. It has been reported that disease severity and mortality rise as the number of SIRS criteria fulfilled increases (19). However, there were no significant differences in APACHE II or SOFA scores, PCT levels, length of stay (LOS) in ICU/ hospital, or 28-day mortality rates of infected patients based on the number of SIRS criteria present at admission. Table 3 shows the details.

SOFA in suspected infection patients
A SOFA score of !2 was recorded in 94% of patients in the first 24 h after infection. SOFA has been used to grade disease severity in many critically ill populations (20). In the present study, we found a gradual and significant increase in APACHE II scores, PCT levels, LOS in ICUs, and 28-day mortality rates corresponding to the increase in SOFA score. Table 4 shows the details.

Patients excluded by sepsis-3
Using sepsis-1, 11 (5.9% infected patients) patients were diagnosed with sepsis but they did not fulfill the criteria of sepsis-3. The characteristics of those 11 excluded patients are listed in Table 5. The culture positive rate of those 11 cases is significantly lower than the sepsis patients who fulfilled both the sepsis 1.0 and sepsis 3.0 criteria (54.5% vs. 93.7%, P ¼ 0.001). The APACHE II and SOFA scores and LOS in ICU of the 11 patients were significantly lower than of patients whose sepsis was defined by both the previous and new criteria. Moreover, the SOFA scores were even lower than that for nonsepsis patients. All 11 patients survived. The mean cost of each patient is 12033.1 AE 7221.8 USD.
Performance of sepsis-1 and sepsis-3 As discussed in the sepsis-3 development paper, there is no gold standard for sepsis diagnoses (7)(8)(9). Therefore, we also evaluated the performance of sepsis-1 and sepsis-3 using a ROC curve to predict 28-day mortality rates. All the decedents had two or more SOFA and 91.4% (117/128 cases) of survivors had less than 2 SOFA points. And all the 186 cases had two or more SIRS  Figure 2. The data regarding overlap between Sepsis-1 and Sepsis-3 indicates that the false positive rate for Sepsis-1 is nearly 50%.

DISCUSSION
The present retrospective multicenter study demonstrated that sepsis-3 performed better than sepsis-1 in China and was comparable with its performance in the primary derived and validated population. The AUROC value of SOFA is significantly higher than that of SIRS when used to predict 28-day    Comparison between sepsis and nonsepsis patients according to sepsis-3. APACHE II indicates acute physiology and chronic health evaluation II; CRP, C-reactive protein; ER, emergency room; ICU, intensive care unit; LOS, length of stay; PCT, procalcitonin; SD, standard deviation; SOFA, sequential organ failure assessment; USD, United States dollar.
mortality rates in infected patients. The ''sepsis-3'' excluded 11 extremely slightly ill cases from ''sepsis patients'' defined by ''sepsis-1.'' The APACHE II and SOFA scores and LOS in ICU of the 11 patients were significantly lower than of patients whose sepsis was defined by both the previous and new criteria. All 11 patients survived, and all received antibiotic treatment. The APACHE II scores, PCT levels, LOS in ICU, and 28-day mortality rates increased gradually and significantly corresponding with the rise in SOFA score but not SIRS score. Despite obviously distinct characteristics between the ICU in the present study and those in the derived settings, there was no impact on the performance of sepsis-3.
In the present study, all the suspected infection patients admitted in ICUs met two or more SIRS criteria, and 11 (5.9% infected patients) less ill patients were still diagnosed as having sepsis according to sepsis-1 but not to sepsis-3. The disease severity of those 11 patients is significantly slighter than that of patients whose sepsis was defined by both the previous and new criteria. And all of them survived. So the characteristics of 11 patients did not support them as real ''sepsis'' patients. However, all those 11 cases were treated as sepsis patients according to surviving sepsis guidelines and eight of them were treated by broad-spectrum antibiotics. All the 11 ''less ill patients'' should not have received antibiotics and those over-treatments have brought considerable burden to our ICUs with limited resource. Only in 1 month on around 100 ICU beds, we spend over 130,000 USD on those ''sepsis'' patients. According to the data from official website of ''Chinese Society of Critical Care Medicine,'' there are around 4,000 ICUs in China in 2010 with around 80,000 beds. It could be estimated that we spend over 1.2 billion USD annually on those ''less ill sepsis'' patients nationally.
The AUROC (95% confidence interval, 0.61-0.77) of SOFA with regard to 28-day mortality rates is 0.69 (95% confidence interval, 0.61-0.77) in the present study, which is comparable to the performance in the primary cohort (AUROC ¼ 0.74 [95% confidence interval, 0.73-0.76]) (9). In ICUs of typical tertiary hospitals in China, all the variables are available and most doctors working in these settings prefer to calculate SOFA scores to evaluate organ dysfunction. Collectively, we validated and recommended SOFA score as the defining criterion for organ dysfunction for sepsis-3 in ICUs in China. qSOFA has been validated in patients with pneumonia in China very recently (21), it was developed to scan for sepsis in non-ICU settings, and, like SIRS, only three items may be challenged by its specificity. SOFA scores have performed well for critically ill patients in China in previous studies. It is valuable to validate qSOFA in the present dataset to confirm the conclusion in the Sepsis-3 datasets or identify another arena where qSOFA might be useful in another arena. However, we only recorded the SOFA scores of every organ system but not the values for qSOFA. We have conducted another investigation to validate qSOFA in ICUs in smaller hospitals in China.
We found that the number of SIRS criteria present could not be used to stratify the severity of illness. Kaukonen et al. (22) also documented the inability of SIRS to stratify the severity of illness. SOFA score has been used as a front-line tool to grade disease severity in ICUs (20,23,24). We investigated the possibility of grading sepsis according to SOFA scores. APACHE II score, LOS in ICU/hospital, and mortality increased as the SOFA score rose. In infected patients, adjusted for age and gender, SOFA is an independent factor contributing to bad outcome. Further studies could grade sepsis as ''mild,'' ''moderate,'' and ''severe'' according to SOFA score, in a way   (25). The present study has its strengths and limitations. Strengths include: the first multicenter study to validate the use of sepsis-3 in ICUs in China and also in low-or mid-income countries; evidence that SOFA score may be used to grade the severity of sepsis. Limitations include: the retrospective design; further prospective studies are needed to confirm the results; only 28days mortality as the endpoint. In the study that developed and validated Sepsis-3 criteria, the outcome assessed was death or an ICU stay > 3 days. But the ICU resources are limited in China, so the strategies that how to admit and discharge from ICUs are quite different compared with developed countries. The length of stay in ICUs is always influenced by many other factors as financial factors, insurance, and limited ICUs beds. We only used death as the endpoint to avoid the deviation; a focus solely on criteria that evaluate the host response to infection, but not on makers to enable precise diagnosis of infections. As reported, between one third and one quarter of patients with signs of clinical infection do not have positive culture results (5-6). We found culture positive infection in 65.1% of 186 patients with infection. Some of the most recently investigated biomarkers, such as elements originating from microorganism or alarmins from the host, may inform development of a further, more precise diagnosis of sepsis in the sepsis-3 era (26).

CONCLUSION
The present study was the first multicenter study conducted in ICUs in China to validate the performance of the new sepsis criteria developed in the US population. We found that sepsis-3 worked well in the study sample. These results may also inform the application of sepsis-3 in various settings or populations, especially in low-and mid-income countries. Further studies are needed to confirm whether SOFA scores can be used to grade the severity of sepsis, which would help to improve the clinical management of sepsis and the allocation of resources in ICUs.