{'fraction_non_alphanumeric': 0.03416524101841602, 'fraction_numerical': 0.029837979269397203, 'mean_word_length': 5.041033434650456, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Validation and refinement of a clinical decision rule for the use of computed tomography in children with minor head injury in the emergency department

BACKGROUND: There is uncertainty about which children with minor head injury need to undergo computed tomography (CT). We sought to prospectively validate the accuracy and potential for refinement of a previously derived decision rule, Canadian Assessment of Tomography for Childhood Head injury (CATCH), to guide CT use in children with minor head injury. METHODS: This multicentre cohort study in 9 Canadian pediatric emergency departments prospectively enrolled children with blunt head trauma presenting with a Glasgow Coma Scale score of 13–15 and loss of consciousness, amnesia, disorientation, persistent vomiting or irritability. Phys icians completed standardized assessment forms before CT, including clinical predictors of the rule. The primary outcome was neurosurgical intervention and the secondary outcome was brain injury on CT. We calculated test characteristics of the rule and used recursive partitioning to further refine the rule. RESULTS: Of 4060 enrolled patients, 23 (0.6%) underwent neurosurgical intervention, and 197 (4.9%) had brain injury on CT. The original 7-item rule (CATCH) had sensitivities of 91.3% (95% confidence interval [CI] 72.0%–98.9%) for neurosurgical intervention and 97.5% (95% CI 94.2%–99.2%) for predicting brain injury. Adding “≥ 4 episodes of vomiting” resulted in a refined 8-item rule (CATCH2) with 100% (95% CI 85.2%–100%) sensitivity for neurosurgical intervention and 99.5% (95% CI 97.2%–100%) sensitivity for brain injury. INTERPRETATION: Among children presenting to the emergency department with minor head injury, the CATCH2 rule was highly sensitive for identifying those children requiring neurosurgical intervention and those with any brain injury on CT. The CATCH2 rule should be further validated in an implementation study designed to assess its clinical impact.

brain injury on CT (Box 1). Decision rule validation is an important step before clinical use. [19][20][21][22] The primary aim of this study was to prospectively validate the accuracy, reliability and acceptability of the CATCH rule in a new cohort of children with minor head injury. The secondary aim was to explore the potential to refine the rule to improve performance.

Methods study design and setting
This was a prospective multicentre cohort study in the emergency departments of 9 Pediatric Emergency Research Canada (PERC) member hospitals from April 2006 through December 2009.

study population
Children (aged 0-16 yr) with acute head injury were eligible for enrolment if they had all of the following: blunt head trauma resulting in witnessed loss of consciousness, amnesia, disorientation, persistent vomiting (≥ 2 episodes of vomiting 15 min apart) or persistent irritability for children 2 years of age or younger; initial emergency department GCS score of 13 or greater; and injury within the past 24 hours. Patients were ineligible if they had obvious penetrating skull injury or depressed fracture; they had acute focal neurologic deficit; they had chronic generalized developmental delay; child abuse was suspected; they returned for reassessment of the same head injury; or they were pregnant. Written informed consent was obtained at the time of enrolment.

standardized patient assessment
Emergency staff physicians or senior residents made all patient assessments. Physicians underwent a 1-hour lecture describing the study and the standardized assessments. After assessing patients but before ordering any imaging, physicians recorded 15 clinical findings on data forms. This included variables in the CATCH rule and other clinically important variables for possible rule refinement. Physicians also documented their interpretation of the CATCH rule and their degree of comfort in using the rule. When feasible, a second physician completed an independent assessment to allow determination of interobserver agreement. Physicians were instructed to use their clinical judgment and not the CATCH rule when deciding when to order CT.
Computed tomography scans were interpreted by staff radiologists, who were blinded to patient assessment forms, at each site as part of routine clinical operations. If the radiologist expressed uncertainty about whether an acute intracranial injury existed, the CT scan was reviewed by another radiologist and a neurosurgeon who were both blinded. If uncertainty still existed, the scan was considered negative.
Because not all children with minor head injury routinely undergo CT, we could not ethically mandate universal CT for all patients. Patients who did not undergo imaging were classified as having no clinically important brain injury if they met all the following criteria at 14 days when contacted by telephone: headache absent or mild; no memory or concentration problems; no seizure or focal motor findings; and return to normal daily activities. These criteria were assessed by a nurse who was blinded to the clinical details of the patient. Patients not fulfilling these criteria were recalled for clinical reassessment and CT. Patients could only be classified as having brain injury based on their CT findings. Patients who did not undergo CT and could not be reached were excluded from the final analysis.

outcome measures
The primary outcome was neurosurgical intervention, defined as either death within 7 days secondary to head injury or the need for any of the following procedures within 7 days: craniotomy, elevation of skull fracture, intracranial pressure monitoring or intubation for head injury (Box 2).
The secondary outcome was brain injury on CT, defined as any acute intracranial finding on CT attributable to acute trauma. This included closed depressed skull fractures (depressed past the inner table) and pneumocephalus, but excluded nondepressed skull fractures and basilar skull fractures.

statistical analysis
We examined demographic and clinical characteristics using simple descriptive statistics. We assessed interobserver agreement with the κ statistic. Univariate analyses used the 2-sided t test for continuous variables and the Pearson χ 2 test for categorical variables. We evaluated the accuracy of the interpretation of the rule by the treating physician versus the criterion interpretation by the investigators. The criterion interpretation of the rule (i.e., whether positive or negative for the outcome measures) was made by an adjudication committee that reviewed patient records and physician responses on data forms.
Potential refinement of the rule was assessed by multivariate recursive partitioning analysis using KnowledgeSEEKER version 8.7. The sensitivity, specificity and 95% confidence intervals (CIs) were Box 1: canadian assessment of tomography for childhood head injury (catch) rule ct of the head is required for children with minor head injury* and any 1 of these findings: High risk for neurosurgical intervention • GCS score < 15 at 2 hours after injury • Suspected open or depressed skull fracture • History of worsening headache • Irritability on examination Medium risk for brain injury on CT • Any sign of basal skull fracture † • Large, boggy hematoma of the scalp • Dangerous mechanism of injury ‡ Note: CT = computed tomography, GCS = Glasgow Coma Scale. *Minor head injury defined as injury within the past 24 hours associated with witnessed loss of consciousness, definite amnesia, witnessed disorientation, persistent vomiting (> 1 episode) or persistent irritability (in a child aged < 2 yr) in a patient with a GCS score of 13-15. †Signs of basal skull fracture include hemotympanum, raccoon eyes, otorrhea or rhinorrhea of the cerebrospinal fluid, and Battle sign.
‡Dangerous mechanism is a motor vehicle crash, a fall from an elevation ≥ 3 ft (≥ 91 cm) or 5 stairs, or a fall from a bicycle with no helmet. calculated for the refined rule. We estimated that a sample size of 4000 patients with minor head injury would be required to provide a sufficient number of brain injury cases to allow a 95% CI of 96%-100% around a sensitivity of 99%. We conducted a post hoc bootstrapping analysis 23 to determine the internal stability of the refined CATCH rule using our previous derivation cohort (n = 3866). The first 1000 samples of the full size of the data set were selected using sampling with replacement. Estimates of the sensitivity and specificity were calculated for each sample, and the mean of these estimates was used as the bootstrap sensitivity and specificity estimates. We used the 2.5 and 97.5 percentile of the 1000 estimates as the lower and upper bound for the 95% CI of the bootstrap estimates. Analyses were conducted using SAS version 9.3.

ethics approval
The research ethics committee of each hospital approved the study. The research ethics board of the Children's Hospital of Eastern Ontario gave the first approval.

Results
Between April 2006 and December 2009, a total of 6525 eligible patients were seen in the 9 study hospitals. Of these, 4494 (68.9%) were enrolled, 1609 (24.7%) were missed eligible patients and 422 (6.5%) did not consent to the study ( Figure 1). Of the 4494 enrolled, 4060 (90.3%) had data forms completed by physicians and complete outcome assessments documented. The remaining 434 (9.7%) patients had no CT scan and no proxy outcome measure as they could not be reached for follow-up by telephone despite multiple attempts. Table 1 shows the clinical characteristics of the 4060 enrolled patients. Patients ranged in age from 1 month to 16 years with a mean age of 9.7 years, and 464 (11.4%) were younger than 2 years. The characteristics of the 1609 missed potentially eligible patients were similar to those of patients who were enrolled, including in mean age (9.5 v. 9.7 yr), male sex (65.6% v. 64.5%), rate of arrival by ambulance (38.4% v. 36.1%), transfer from another hospital (13.1% v. 12.5%) and mechanism of injury. Table 2 reports the management and outcomes of the enrolled patients. A total of 1417 (34.9%) patients underwent CT, and the remaining 2643 (65.1%) patients, who were all discharged from the emergency department, underwent the 14-day telephone follow-up. Of those, 20 patients were recalled by study nurses, and 19 of these patients underwent CT, which were all negative. Of all cases, 23 (0.6%) patients underwent neurosurgical intervention (with 20 receiving a craniotomy), and 197 (4.9%) patients were found to have a brain injury on CT. Table 3 shows the performance of the predictor variables in patients with and without brain injury, as determined by univariate analyses and interobserver agreement. All 7 variables of the CATCH rule were highly associated with acute brain injury (p < 0.001). Other variables strongly associated with brain injury included repeated vomiting (≥ 3 and ≥ 4 episodes), initial GCS score of 13, scalp hematoma and lethargy on examination. The κ value for the overall interpretation of the CATCH rule was 0.67 (95% CI 0.60-0.74).
The classification performance of the CATCH rule is shown in Table 4. For detecting patients requiring neurosurgical Box 2: definitions of traumatic brain injury outcomes Primary outcome: neurosurgical intervention* Defined by any of the following descriptions: • Death within 7 days due to traumatic brain injury • Need for any of the following procedures within 7 days: • craniotomy • elevation of depressed skull fracture • intracranial pressure monitoring • intubation for the treatment of head injury secondary outcome: brain injury on ct † Defined by any of the following descriptions: Note: CT = computed tomography. *The neurosurgical intervention group requires prompt recognition and an intervention to treat the traumatic brain injury. †The brain injury on CT group usually requires admission to hospital for observation and neurologic follow-up. intervention, the sensitivity was 91.3% and the specificity was 57.1%. For detecting patients with brain injury on CT, the sensitivity was 97.5% and the specificity was 59.6%. Rule refinement by recursive partitioning showed that the best model to predict all cases of neurosurgical intervention and all but 1 case of brain injury on CT retained all the variables in the original 7-item CATCH rule and added the variable "vomiting ≥ 4 episodes." This variable describes the number of episodes of vomiting (> 15 min apart) that occurred since the head trauma. The classification performance of this new rule is shown in Table 5. Sensitivity for detecting patients requiring neurosurgical intervention using the new rule was 100% (95% CI 85.2%-100%) and the specificity was 45.7% (95% CI 44.2%-47.3%). Sensitivity for detecting brain injury on CT was 99.5% (95% CI 97.2%-100%) and the specificity was 47.8% (95% CI 46.8%-49.4%). We designated this refined 8-item rule the CATCH2 rule, shown in Box 3.
The potential impact of the rule was assessed by comparing the CT rate according to the CATCH rule to the actual clinical practice of the physicians at the 9 sites. For the 4060 enrolled patients, the CT rate according to the CATCH rule was 43% and the CT ordering rate by the physicians was 35%. Using the CATCH2 rule, the CT ordering rate would be 55%.
One case of brain injury would have been missed using the CATCH2 rule. This was an adolescent whose occiput struck the ground after a tackle. The patient had loss of consciousness for 2 minutes, amnesia to the event and a moderate headache (not worsening). Computed tomography on presentation to the emergency department showed a 1 mm thick acute subdural bleed. The patient was discharged directly home from the emergency department after observation.
Physicians reported they would be comfortable to very comfortable using the rule in 3295 cases (81.5%), neutral in 347 cases (8.6%), and uncomfortable to very uncomfortable in 399 cases  Death secondary to head injury 0 Note: CT = computed tomography. *Some patients had more than 1 lesion. †Some patients had more than 1 intervention.
(9.9%). In an evaluation of accuracy, treating physicians misinterpreted the rule as not requiring imaging when the clinical criteria suggested imaging in 8.7% of patients. The most frequently misinterpreted variables were dangerous mechanism of injury and irritability.

Interpretation
This validation study shows that the 7-item CATCH rule had inadequate sensitivity to detect those children requiring neurosurgical intervention. The rule was refined by adding a variable for recurrent vomiting to create the 8-item CATCH2 rule, which achieved 100% sensitivity for neurosurgical intervention and 99.5% sensitivity for detecting brain injury on CT. The trade-off for this improved sensitivity is decreased specificity. Following this rule would result in 55% of patients with minor head injury undergoing CT. The variables that make up the CATCH2 rule have been found to be significant predictors of intracranial injury in previous observational studies of pediatric head injury 24-29 including a GCS score of less than 15 at 2 hours, [25][26][27][28][29] suspected open or depressed skull fracture, [25][26][27][28][29] worsening headache, 24,25 persistent irritability, 26 sign of basal skull fracture, [25][26][27][28][29] large boggy scalp hematoma, 26,27 recurrent vomiting 26,27,29 and dangerous mechanism. [27][28][29] This consistency between studies supports the importance of these risk factors in predicting intracranial injury in children.
The Pediatric Emergency Care Applied Research Network (PECARN) developed decision rules to identify children at low risk of clinically important traumatic brain injury who could be safely sent home without CT. 29 Although use of these decision rules may reduce the use of CT in children with minimal symptoms, these rules do not indicate which children with more concerning signs and symptoms require CT. Interestingly, the PECARN study does identify "altered mental status" and "signs of skull fracture" as the highest risk clinical factors of clinically important traumatic brain injury, which is consistent with 5 variables of the CATCH2 rule.
Babl and colleagues 30 recently published a well-designed prospective external validation of the CATCH, 6 PECARN 29 and CHALICE (Children's Head Injury Algorithm for the Prediction of Important Clinical Events) 27 rules. They reported a high sensitivity (95.2%, 95% CI 76.2%-99.9%) for the CATCH rule, which predicted 20 of 21 patients requiring neurosurgical intervention. The 1 missed patient had repeated vomiting (> 3 times) and would have been identified by the new CATCH2 rule had it been available for their validation study.
We found that the rate of CT use for children with minor head injury has decreased significantly between the time of our derivation study (2001)(2002)(2003)(2004)(2005), when the rate was 53%, and our validation study (2006)(2007)(2008)(2009), when the rate was 35%. 6 We believe that raised awareness of the risks associated with radiation 10,11,13 involved with CT has already resulted in pediatric emergency physicians becoming more selective with whom they choose to undergo CT.

limitations
This study has potential limitations. Computed tomography was not performed on all enrolled patients because we could not ethically justify exposing children to radiation when their treating clinician felt CT was not required. However, the patients who did not receive CT underwent a validated telephone interview at 14 days for determination of the proxy outcome measure. Although 9.7% of patients were lost to follow-up, this rate is similar to those of other large prospective emergency department studies of head injury in children, such as the studies by Kuppermann and colleagues 29 and Babl and colleagues, 30 which had lost-to-follow-up rates of 21% and 10%, respectively. Although the interobserver agreement (κ = 0.67) for the CATCH rule is considered moderate to substantial, [31][32][33] future implementation studies should focus on improving this. Bootstrapping procedures for exploring uncertainty in model development of the CATCH2 rule may be informative but could not be followed because the specific classification and regres-sion tree approach used for developing this rule does not lend itself to repeated model building. We did apply the CATCH2 rule to the derivation data set using a bootstrapping technique to provide a more robust evaluation of the sensitivity and specificity of the rule and to show that the slight revision is not of such magnitude to invalidate the rule. As expected, the revised rule was validated. However, the formal verification of the validation of the CATCH2 rule will come through applying it to a new prospective data set during the subsequent implementation phase.
The decision to obtain a CT scan is affected by many factors, including setting, number and progression of symptoms, patient and parent preference, and physician experience. Although the sensitivity of the CATCH2 rule is very high, it is not perfect. The rule was developed according to strict methodologic standards 19,20 to assist physicians in making decisions regarding immediate CT, prompt referral to a centre with CT availability or need for continued close observation in the emergency department. The effect on CT rates will depend on the emergency department setting and local practice patterns; however, using this rule should result in exceedingly few missed injuries.
conclusion Among children presenting to the emergency department with the signs and symptoms of acute minor head injury, the CATCH2 rule was highly sensitive for identifying those children requiring neurosurgical intervention and those with any brain injury on CT. The CATCH2 rule should be further validated in an implementation study designed to assess its clinical impact.