{'fraction_non_alphanumeric': 0.03080482162425423, 'fraction_numerical': 0.020495961686756766, 'mean_word_length': 5.385073853329878, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Comparing Expert and Non-Expert Assessment of Patients Presenting with Neurological Symptoms to the Emergency Department: A Retrospective Observational Study.

Objective
Referrals to neurology in emergency departments (ED) are continuously increasing, currently representing 15% of all admissions. Existing triage systems were developed for general medical populations and have not been validated for patients with neurological symptoms.


Methods
To characterize neurological emergencies, we first retrospectively analyzed symptoms, service times and resources of the cohort of neurological referrals to a German interdisciplinary ED (IED) during 2017 according to urgency determined by final IED diagnosis. In a second step, we performed a retrospective assignment of consecutive patients presenting in April 2017 according to internal guidelines as either acute (requiring diagnostic/therapeutic procedures within 24 hrs) or non-acute neurological conditions as well as a retrospective classification according to the Emergency Severity Index (ESI). Both assessments were compared with the urgency according to the final ER diagnosis.


Results
In a 12-month period, 36.4% of 5340 patients were rated as having an urgent neurological condition; this correlated with age, door-to-doctor time, imaging resource use and admission (p < 0.001, respectively). In a subset of 275 patients, 59% were retrospectively triaged as acute according to neurological expertise and 48% according to ESI categories 1 and 2. Neurological triage identified urgency with a significantly higher sensitivity (94.8, p < 0.01) but showed a significantly lower specificity (55.1, p < 0.05) when compared to ESI (80.5 and 65.2, respectively).


Conclusion
The ESI may not take specific aspects of neurological emergency (eg, time-sensitivity) sufficiently into account. Refinements of existing systems or supplementation with dedicated neurological triage tools based on neurological expertise and experience may improve the triage of patients with neurological symptoms.


Introduction
Emergency conditions represent a large portion of the global burden of disease, and emergency department (ED) utilization in many countries has substantially increased in recent years. 1,2 In Germany, for instance, the number of referrals has risen to approximately 20 million over the past several years and is expected to increase further. 3 Notably, there has been a substantial increase in ED patients with neurological disturbances and disorders, currently accounting for 15% of ED admissions and up to 26% of medical patients in general. [4][5][6] Treatment of patients with neurological emergencies such as acute ischemic stroke, cerebral hemorrhage, status epilepticus or meningoencephalitis is often time-sensitive and requires swift management in the ED. [7][8][9][10] In many cases, the character and acuity of the presenting neurological disturbances clearly indicate the necessity for rapid diagnostic and therapeutic procedures in an ED setting. Often, however, the handling of patients with neurological signs and symptoms can be challenging for prehospital first-aid and non-neurological emergency care providers because symptoms may be non-specific and related to a wide spectrum of differential diagnoses and hence different degrees of urgency. 11 Triage systems were developed to facilitate acuity assessment and to predict patient disposition and resource utilization. The Emergency Severity Index (ESI) is one of the most commonly used formal five-level triage systems developed to estimate acuity and resource consumption. The ESI system rates patients from level 1 (most acutely ill) to level 5 (least resource-intensive) and has been shown to have a high inter-observer agreement and to predict hospital admission and location of admission. 12,13 To date, however, there is no dedicated validation study of existing triage systems to assess their performance in neurological patients, nor exists a dedicated triage system for patients with neurological complaints.
Our aims in this retrospective study were, first, to characterize neurological emergencies at presentation to an interdisciplinary ED (IED) and, second, to evaluate the sensitivity and specificity of the ESI compared to an expert neurological assessment of these patients.

Study Design
We retrospectively analyzed patient records of 5340 patients who consecutively presented or were referred to the IED of the University Medical Centre, Mannheim, Germany, between January 1st 2017 and December 31st 2017 for neurological consultation. The study protocol has been approved by the Ethics Committee II, Medical Faculty Mannheim, Heidelberg University, reference number 2018-502N-MA, and has been performed with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. Due to the retrospective design and lack of study-related interventions, no consent to participate was obtained. The manuscript does not contain clinical studies or patient data.
In the Mannheim IED, at least one neurology resident is present 24/7 either for first-line assessment if prehospital assessment or evaluation on arrival suggests a neurological condition, or second-line if first-line assessment by a nonneurologist IED physician suggests a neurological condition. Cases were identified by filtering the ED database for patients coded to present with a chief complaint that was evaluated by emergency medical service or ED staff upon arrival to be a neurological symptom or complaint.

Analysis of Neurological Referrals to the Interdisciplinary ED
Data extracted from patient records include basic demographic information, chief complaint/presenting symptom categorized according to Royl,14 door-to-doctor time, length of IED stay and resource utilization (computed tomography (CT)/angiography (CTA), magnetic resonance imaging (MRI), lumbar puncture (LP), electroencephalography (EEG)) during patients' stay in the IED. Based on information from diagnostic procedures and the final IED diagnosis, patients were retrospectively classified as having either an urgent neurological condition or a non-urgent neurological condition, representing the gold standard of "true" urgency. 15 In accordance with Barbadoro, an urgent neurological condition was defined as any condition requiring in-hospital equipment for diagnostic or therapeutic purposes immediately or within the next 24 hrs to avoid harmful consequences for the patient. 16 A non-urgent neurological condition, in contrast, refers to any neurological condition not meeting these requirements, eg, those suitable for further work-up in an outpatient setting. We analyzed and compared the characteristics of the two groups.

Retrospective Non-Formalized Expert Neurological Evaluation versus ESI
In a second step, records of a subset of 299 patients consecutively presenting to the ED during a 4-week period were retrospectively evaluated for level of acuity by two experienced neurologists with more than 4-year practical experience in the IED: only the history pertaining to the chief complaint and symptoms, any additional pre-existing diagnoses and current medication were presented, on the basis of which it was determined whether the patient was thought to require diagnostic or therapeutic procedures within the next 24 hrs (acute), or whether this was a condition not necessitating such procedures (non-acute). This decision was made with reference to internal guidelines which were established with close reference to official guidelines of the German Neurological Society for the management of neurological conditions. These guidelines have been in use for several years and are continually checked and updated. In addition, retrospective ESI ratings were performed. Sensitivity, specificity, positive and negative predictive values for the non-formalized neurological evaluation and ESI identification of acute cases ("ESI acute" subsuming ESI categories 1 and 2) against the gold standard urgent/non-urgent were calculated. For an overview of methods, see Figure 1.

Statistical Analysis
Statistical analysis was performed with IBM SPSS Statistics version 22.0. The distribution of categorical variables between urgent and non-urgent patients was compared by Chi2 tests. Group comparisons of ordinal data were assessed using Mann-Whitney-U-tests and group comparisons of metric data were assessed using independent samples t-tests. Bonferroni correction for multiple testing was applied where suitable. In the tables and in the text unadjusted p-values are given, but in the case of multiple testing, these are interpreted as significant only if still significant after Bonferroni correction. McNemar's Chi2 test was used to compare performance measures (sensitivity and specificity) of ESI and expert neurological assessment. 17 A p-value of <0.05 was considered significant.

Patient and Public Involvement
Patients and/or public were not involved in this study.

Characteristics of Neurological Referrals to the IED
Within the 12-month observational period, a total of 45,445 patients were seen in the IED. Of these, 5340 (12.0%) patients were evaluated by a neurologist. Mean age of all neurologically evaluated patients was 56.2 years (SD ±21.27), 2583 (48.4%) were male. We identified 1896 patients (36.4%) with an urgent neurological condition and 2427 patients (46.6%) with a non-urgent neurological condition (Table 1). Moreover, 890 patients presented with a neurological chief complaint but ultimately did not have a neurological diagnosis. The non-urgent population comprised significantly more patients of up to 65 years of age, while in the urgent group, there were significantly more patients over 65 years of age (p<0.001). Patients with an urgent condition were brought to the IED by paramedics and/or an emergency physician significantly more frequently than non-urgent patients (n= 1283 [67.7%] vs n=1192 [49.1%], p<0.001), whose presentation was more often selfinitiated, or they were referred by a practice-based physician (n=1233 [50.8%] vs n=611 [32.2%], p<0.001). Overall, vertigo (n=793, 14.9%), motor deficits (n=672, 12.6%), headache (n=663, 12.4%) and epileptic seizures (n=590, 11.0%) were the most frequent chief complaints. Patients over 65 years of age most frequently presented with motor deficits (n=374), aphasia/dysarthria/dysphagia (n=369) and vertigo (n=270), while headaches (n=588), vertigo (n=526) and epileptic seizures (n=444) were the major causes for ED visits of younger patients. Disturbed consciousness, motor deficits, vision disturbances as well as aphasia/dysarthria/dysphagia were significantly more frequently found as presenting symptoms of urgent patients (p<0.001). On the other hand, epileptic seizures, headaches and vertigo were more common in non-urgent patients (p<0.001). Even though certain chief complaints were more common in patients presenting with an urgent condition versus those with a non-urgent condition and vice-versa, chief complaints alone did not sufficiently Figure 1 Flowchart of study analysis. First, clinical characteristics and medical procedures of all neurological referrals to the IED during 2017 were analyzed and compared for urgent and non-urgent cases according to the final diagnosis. In a second step, an informal triage in 299 of these patients was performed according to our internal guidelines as either acute or non-acute. This was compared with a classification according to the ESI. Abbreviations: IED, interdisciplinary emergency department; ER, emergency room; ESI, Emergency Severity Index.
identify patients categorized as urgent and requiring immediate medical assessment. While urgent patients were seen by a doctor faster (33.7 vs 43.7 min, p<0.001), the length of stay in the IED did not differ between patients with an urgent vs a non-urgent neurological diagnosis (253.5 vs 250.1 min, p=0.528). Overall, 2215 patients (51.2%) were admitted. Significantly more urgent patients (n=1764, 93.0%) than non-urgent patients (n=451, 18.6%) were admitted to hospital (p<0.001). A detailed description of the study population is presented in Table 1.

Retrospective Neurological Evaluation and Triage According to ESI
Of a subset of 299 patients consecutively admitted to our IED during a 4-week-period, 24 were diagnosed as not neurological and were excluded from further analysis. The mean age of the remaining 275 patients was 55.05 years (SD ± 21.36 years), 128 (46.5%) were men. Of the 275 cases, retrospective neurological evaluation rendered 162 patients (58.9%) as needing immediate neurological treatment or diagnostic procedures and 113 (41.1%) as not needing immediate medical attention. The reason for acuity was a) time-sensitive diagnostic procedures or treatment in 141 cases (51.3%), b) the need for immediate and continuous monitoring in 18 cases (6.5%), and c) treatment or monitoring not suited for an outpatient setting in nine cases (3.3%), respectively. In the group of cases rated acute, the most common final diagnoses were acute ischemic stroke (n=37, 22.8%) and seizures (n=24, 14.8%). In those cases that were rated as non-acute, the most common final diagnoses were headache and seizures (each n=22, 19.5%). Of the 113 nonacute patients, a relevant previous neurological diagnosis was known in 26 patients (23.0%), mostly a previously diagnosed epilepsy (n=16, 14.2%).
The number of admissions and utilization of specific neurological resources (CCT/CTA, MRI, LP) by retrospective neurological assessment and ESI categories is presented in Table 2. The distribution of ESI categories 1-5 compared to the neurological assessment as acute or non-acute is depicted in Figure 2. The distribution of cases as acute/nonacute according to neurological evaluation and as ESI acute (subsuming ESI categories 1 and 2)/ESI non-acute (subsuming ESI categories 3 to 5) across a spectrum of diagnoses is depicted in Figure 3. Specificity, sensitivity, positive and negative predictive values of the non-formalized neurological evaluation as acute vs non-acute and ESI acute are presented in Table 3. The expert neurological evaluation was significantly more sensitive (p<0.01) while ESI (in the previously mentioned dichotomous distinction "ESI acute" subsuming ESI categories 1 and 2 vs "ESI non-acute" subsuming categories 3 to 5) was significantly more specific (p<0.05).

Discussion
In our retrospective analysis of patient characteristics and medical procedures of neurological patients in an IED of a university hospital, only slightly more than 50% of patients required hospitalization. In addition, a retrospective   non-formalized neurological expert evaluation was more sensitive but less specific than ESI. The ESI is one of the most widely used triage algorithms and supports decision-making by taking information regarding clinical risk status and expected resource utilization into account. 12 Recently, criticism was raised regarding a lack of reliable discrimination, leading to a high percentage of patients triaged to ESI category 3. 18,19 In our cohort of patients with neurological signs and complaints and confirming data reported by Lange and coworkers, 20 we found a considerable number of patients receiving ESI category 2, because patients with seizures and suddenly occurring focal deficits regardless of onset are triaged to this level. From a neurological perspective, there is a highly relevant difference in acuity of a patient with a focal neurological deficit that has been present for 2 hrs or for 12 hrs, or between a patient with a first-ever seizure or a recurrent seizure and a history of epilepsy.
Patients with highly acute neurological diseases formally do not qualify for the highest urgency level in ESI, as they rarely present in states requiring immediate life-saving interventions. In addition, vital signs, which in established triage systems take precedence over the time-is-brain concept, 20 in most cases do not convey neurological urgency. Cerebrovascular disease excellently illustrates this aspect: therapeutic options are time-limited and even long-term prognosis depends on the actual timing of treatment. In acute ischemic stroke, the sooner treatment with the thrombolytic agent rt-PA is administered to patients, the greater the benefit, especially if started within 90 mins following symptom onset. 21 A waiting time of 10 mins for a patient with an acute neurological deficit as formally granted by many triage systems and subsequent delay in diagnosis and treatment may thus negatively impact patient outcome. In recent years, a variety of interventions have been investigated to improve the early management of patients with suspected stroke, ranging from the development of stroke recognition tools to the establishment of stroke codes on pre-notification or upon arrival. [22][23][24][25] These procedures have been highly effective, and it is common practice to incorporate them into institutional policies so that they may override the acuity assignment as formally specified by triage. However, it is unclear whether stroke codes are also activated when patients present with mild or atypical symptoms, which have been shown to lead to more frequent assignments to non-acute ESI categories with the ensuing risk of delayed diagnosis and therapy. 26,27 Patients presenting with dizziness or vertigo are also assigned to lower acuity categories unless vital signs are alarming. Subtler abnormalities, eg, of the oculomotor system, that are suggestive of a central nervous system pathology may be missed in the initial assessment of these patients by the triage nurse or non-neurologists. 28 Since dizziness is a frequent complaint in patients presenting to the emergency department, the detection of the small portion of patients with a serious underlying etiology is particularly challenging. A timely and accurate bedside diagnosis is required in the interests of patient safety but also economically, given the enormous costs caused by inappropriate or unnecessary testing.
Another aspect of the problem is illustrated by the fact that approximately half of all patients evaluated by a neurologist ultimately did not require hospitalization. Hospital admission rates from the ED are influenced by a variety of factors including patient age and affected organ system, with admission rates for general medicine, vascular and neurosurgery patients generally being higher than those for neurological patients. 29,30 For this patient group, a previous study found a hospital admission rate from the ED of 58.1%. 29 In a study by Robertson et al, only one-third of patients referred for neurological assessment to a rapid referral neurological acute clinic were retrospectively considered to have warranted an urgent evaluation. 31 On a related note, providing a diagnosis rather than obtaining therapeutic recommendations has been identified as the main motivation for in-house neurological consultation requests in a recent Spanish study. 32 Insufficient experience, the fear of missing a critical time-window for therapy initiation and resulting safety-thinking may lead non-neurological medical care providers to initiate emergency assessments sooner rather than later even with non-specific symptoms or benign conditions. 33 While such an approach generally appears to favor patient safety, it may result in over-testing and overdiagnosis yielding adverse consequences of their own. 34 In general, patients not requiring urgent diagnosis or treatment substantially contribute to the increase of ED presentations observed in recent years. 35 The potential 24/7 availability of resources versus difficulties receiving prompt appointments with non-hospital neurologists and the perceived convenience of access to different medical specialties in particular motivate patients to visit the ED for non-urgent conditions. 36,37 This development has prompted the search for alternative solutions, 38,39 those specifically addressing neurological patients include email triage and urgent assessment clinics. 40 However, many of these options will not be able to provide immediate evaluation, which still leaves a considerable portion of patients seeking help in EDs. Accordingly, methods of formal and objective assessment of urgency as well as ED suitability are required, reverberating in an extended understanding of triage incorporating aspects of demand management. 41 Taking these facets of neurological emergency care into account, triaging patients with neurological complaints is challenging: On the one hand, algorithms should be sensitive enough to capture true emergencies even in cases of atypical or subtle presentations, and they need to be specific so as to facilitate the adequate and targeted allocation of limited resources. The neurological expert assessment was demonstrated to be more sensitive than ESI in the detection of urgent neurological conditionsthe neurological understanding of what constitutes an emergency certainly contributes to this finding. ESI, on the other hand, was more specific. We do not argue, however, that triage of patients with neurological complaints should be performed by neurologists. Our investigation should rather be taken as an incentive to refine their representation in current triage systems in order to account for their idiosyncrasies and thus to continually improve patient care.
Our findings should be interpreted in the context of some limitations. First, this study is a retrospective cohort study, and as such, subject to the characteristic limitations of using retrospective data, especially as it critically relies on the accuracy of patient records obtained from the hospital electronic database. Second, the study was conducted as a single-center analysis; therefore, the results might not be generalizable to other IEDs with different patient populations and organizational structures. The retrospective evaluation of acuity and urgency also possesses inherent limitations. To begin with, the assessment of acuity in our study relied on written information. Since both analytical and intuitive processes impact on clinical decision-making, any retrospective evaluation, lacking the experiential component, will most likely be skewed. 42 Moreover, it is difficult to retrospectively evaluate the necessity of a diagnostic test, particularly when negative. On a related note, the identification of "true" urgency, which is required to assess the performance of a triage system, is inherently problematic. 43 Finally, we did not perform a detailed analysis of patients who presented with non-neurological symptoms but eventually received a neurological diagnosis. While undoubtedly of high interest and value, we focused on neurological chief complaints as presenting symptoms.

Conclusion
To date, there is no specific triage instrument for neurological complaints, and no studies have addressed the suitability of established triage instruments for neurological assessment. Our data show that a neurological expert assessment is more sensitive than ESI. Our finding of a lower specificity on the other hand indicates that the presentation of neurological conditions can often be non-specific or ambiguous. Both aspects can be best met by providing neurological expertise as early as possible. In light of recent developments in the treatment of ischemic stroke with extended time-frames for acute intervention, first-line neurological presence and experience is needed in the emergency room and will become increasingly relevant. 44 A crucial step for future improvements of ED neurological evaluation concerns a sufficient appraisal of symptom acuity and temporal evolution and the acknowledgement of the syndromal character of many neurological conditions. This may be done by refining current triage systems through a more elaborate representation of neurological symptoms. Moreover, the incorporation of aspects of demand management may also contribute to improvement in care, for example, through the establishment of alternative models of out-patient neurological care to relieve ED overcrowding.

Author Contributions
All authors contributed to data analysis, drafting or revising the article, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Disclosure
Prof. Dr. Simon Nagel reports personal fees from Brainomix, personal fees from Pfizer, grants from Cerenovus, personal fees from Böhringer Ingelheim, outside the submitted work. The authors report no other conflicts of interest in this work.