{'fraction_non_alphanumeric': 0.03684655488234465, 'fraction_numerical': 0.022051571086374525, 'mean_word_length': 5.1238136324417605, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Advanced Practice Provider Initiative for Delirium Management in the ICU

Abstract Delirium is the most common neuropsychiatric challenge in cancer patients, particularly in the critically ill population. Without a screening method and constant vigilance by providers, delirium is often misdiagnosed. The purpose of our pilot study was to determine if an educational program targeting critical care medicine advanced practice providers (APPs) and fellows in an oncologic intensive care unit would increase APP knowledge of delirium and their level of comfort with delirium screening and management. Thirty-one APPs and fellows participated in the program. Scores on knowledge-based delirium assessments increased significantly after the intervention and at the 3-month follow-up (p < .0001 and p < .0225, respectively). Providers’ comfort with delirium screening and management also improved after the intervention (p = .0020 and p < .0001, respectively) and decreased slightly at the 3-month follow-up (p = .1764 and p = .9840, respectively). A brief and focused APP-led educational initiative successfully improved knowledge of delirium and comfort with screening and management.

D elirium is an acute disturbance in cognition, attention, and awareness that may fluctuate throughout the course of the day and is often due to an evolving medical condition (American Psychiatric Association, 2013). Delirium occurs in 30% to 80% of patients admitted to the intensive care unit (ICU) and is associated with increased ICU and hospital mortality, ICU and hospi-tal length of stay (LOS), and healthcare costs (Ely et al., 2001;Girard, Pandharipande, & Ely, 2008;Leslie, Marcantonio, Zhang, Leo-Summers, & Inouye, 2008;Mehta et al., 2015;Shehabi et al., 2010). Delirium carries short-and long-term sequelae, such as decreased ability to perform activities of daily living and varying degrees of cognitive impairment (Brummel et al., 2014;Pandharipande et al., 2013;van den Boogaard et al., 2012).
Delirium is the most common neuropsychiatric complication in cancer patients, with an incidence of 18% to 50% in all patients, up to 43% in patients who have had hematopoietic stem cell transplants, and up to 80% in patients with advanced cancer (Beglinger et al., 2006;Sánchez-Hurtado et al., 2017;Traube et al., 2017). Cancer patients have additional risk factors for delirium that include the direct neurocognitive effects of cancer, chemotherapy, immunotherapy, and radiation therapy, the nature and complexity of certain surgical interventions, and side effects of various supportive medications (anticholinergics, antiemetics, opioids, and anxiolytics;Breitbart & Alici, 2012;Choi et al., 2017).
Untrained health-care providers routinely miss the diagnosis of delirium more than two thirds of the time (Sinvani, Kozikowski, Pekmezaris, Akerman, & Wolf-Klein, 2012). Oncologists have been shown to misdiagnose delirium in up to one third of patients (Wada, Wada, Wada, & Onishi, 2010). Diagnosing delirium in cancer patients can be challenging. Cancer patients are prone to mood disorders, anxiety, and cognitive impairment (Edelstein & Alici, 2017). Critically ill and/or mechanically ventilated patients have a limited ability to communicate (Ely et al., 2001).
The Confusion Assessment Method for the ICU (CAM-ICU) is a well-validated tool for delirium screening (Luetz et al., 2010). CAM-ICU has moderate to high sensitivity (81%) and high specificity (98%) when used by experienced ICU nurses (Shi, Warren, Saposnik, & MacDermid, 2013). Gaps in delirium knowledge, transient and brief clinical assessment, and limited application of CAM-ICU as a screening tool may explain the low rate of recognition of delirium by ICU physicians and possibly by advanced practice providers (APPs; Arumugam et al., 2017). Data on delirium management by APPs are lacking. The purpose of this study is to determine if a brief, ICU-based educational program targeting critical care medicine (CCM) APPs and fellows increases knowledge of delirium and improves the level of comfort with screening and managing delirium.

METHODS
The intervention took place in a 498-bed academic tertiary care cancer center in New York, New York. The 20-bed ICU averages 1,100 admissions per year and is managed by two multidisciplinary teams. The teams consist of attending physicians, CCM fellows, anesthesia residents and interns, and APPs. The APPs are comprised of nurse practitioners (NPs) board-certified in family, adult, and acute care, and board-certified physician assistants (PAs). A group of ICU-based APPs created a comprehensive training program targeting NPs, PAs, and CCM fellows working in the ICU.
The evidence-based delirium training program incorporated current literature on delirium and standardized screening tools. The program consisted of a PowerPoint lecture focusing not only on delirium pathophysiology and its devastating impact, but also on proper delirium screening, diagnosis, and management. A brief video demonstrating CAM-ICU administration was provided, and the 1-hour sessions concluded with time for questions and discussion (Jones, 2013). The educational sessions were led by one of three dedicated acute care APP trainers. The sessions took place during normal working hours to accommodate both the day and night shift staff. Up to five participants attended each session, and the APP trainers completed the program in approximately 2 months (Appendix A).
The CCM team developed a 25-question survey to assess the knowledge of the participants at three time points. Prior to our intervention, the survey was administered to psychiatry and geriatric specialists for review. Based on their feedback, the questions were adjusted for comprehension and readability. We did not create the survey using formal development procedures. The survey was comprised of three segments: • 8 questions that recorded participants' demographic data, professional experience, and familiarity with CAM-ICU and delirium • 15 questions that captured participants' knowledge of delirium diagnosis and management (Appendix B) • 2 subjective questions with a numerical (0 to 100) Likert scale to evaluate participants' reported comfort with CAM-ICU administration and delirium management. The survey was administered to each participant at three different time points: before the training session, immediately after the training session, and 3 months later. The responses from each survey participant were collected on the Research Electronic Data Capture (REDCap) database (Harris et al., 2008). The study was performed for quality improvement, and a waiver of authorization was obtained from the institutional review board before performing the data analysis. Questions were analyzed to identify the most frequently incorrect and correct responses.
Results are presented as medians with first and third interquartile ranges (IQR), means and standard deviations (SD), absolute numbers, and percentages. To compare results of the assessments, Wilcoxon signed-rank test was used for continuous variables, and McNemar's test was used for binary variables because of the paired nature of the data (Table 1). False discovery rate was used to adjust for multiple testing (Benjamini & Hochberg, 1995). When comparing the data collected at three different time points, the factors of interest were the total number of correct answers out of the 15 knowledge questions and reported comfort level with CAM-ICU administration and delirium management based on the Likert scale (0-100), with a score of 100 being the most comfortable. SAS version 9.4 was used for analysis. All tests were twosided, and p < .05 was considered significant.

Participants
The training program targeted 8 CCM fellows, 20 board-certified NPs, and 3 board-certified PAs. The median years of overall ICU experience was 6 (range: 2.5-10). The median years of ICU experience at our institution was 2.5 (range: 1-8). Of the 31 participants, 21 (68%) felt that their CAM-ICU documentation was accurate prior to our training program; however, 13 (42%) had never used CAM-ICU in practice and 26 (86%) had never attended an ICU-wide educational program for delirium screening and management. Additionally, there were no formal educational interventions for APPs or CCM fellows in proximity to our educational intervention.

Knowledge Scores
After the initial educational program, mean scores on the 15-item survey improved from a baseline of 11.5 out of 15 (76%) to 12.9 out of 15 (86%; p < .0001). Three months following the intervention, scores were consistently higher than pre-test scores (12.2 vs. 11.5, p = .0225) and only slightly lower than postintervention scores (12.2 vs. 12.9, p = .0315; Table  1). After analyzing each individual question, the question answered incorrectly the most often was related to the use of haloperidol for the management of delirium, with only 32% of participants answering correctly on the pre-test, 39% on the post-test, and 26% on the follow-up test (question 5 in Appendix B). Additionally, one question required the participants to recognize delirium as a serious syndrome with a major impact on patients' morbidity and mortality. Yet, only 61% of our APPs and fellows answered the question correctly on the pre-and post-intervention quizzes (question 1 in Appendix B).

Comfort Level
Participants' reported comfort with CAM-ICU administration increased from a median Likert score of 51 out of 100 prior to our intervention to 72 out of 100 following the intervention (p < .0001). Similarly, comfort level with delirium management  Note. a p values provided are adjusted using false discovery rate.
increased from a baseline median score of 61 out of 100 to 73 out of 100 after the initial training session (p = .0020). On the other hand, comfort with administering the CAM-ICU decreased slightly from a median score of 72 after the initial training session to 70 at the 3-month follow-up, while comfort in managing delirium increased from 73 immediately after the training program to 76 at the 3-month follow-up. These changes in the level of comfort with CAM-ICU administration and delirium management from training to follow-up were not statistically significant (p = .9840 and p = .1764, respectively).

DISCUSSION
This study demonstrates the successful implementation of an APP-led program that has improved CCM providers' knowledge of delirium screening and management in a mixed medicalsurgical ICU despite a variable level of experience among the participants. Limited prior participant exposure and familiarity with the CAM-ICU screening tool and delirium management did not negatively impact the outcomes of the training program.
Most participants persisted in their incorrect response to question 5 (Appendix B) on the pharmacologic management of agitation associated with delirium throughout the three time point assessments. While the study was not designed to investigate the reasons for correct or incorrect responses, we attribute these consistent errors to (1) gaps in knowledge regarding pharmacologic treatment of delirium, (2) providers' potential inherent bias against the use of haloperidol, (3) poor phrasing of the question, and/or (4) poor explanation by the APP trainers on the correct use of this medication for the treatment of delirium.
The fact that participants persisted in their incorrect response to question 1 (Appendix B) on recognizing delirium as a medical emergency points to (1) the inherent beliefs that delirium is an expected and nonpreventable phenomenon in critical care and/or cancer patients and (2) the limits of singular educational interventions in changing archetypal misconceptions and attitudes anchored in decades of erroneous traditions.
We hypothesize that participants' comfort with the CAM-ICU screening tool and delirium manage-ment initially improved after the implementation of our education program for two major reasons: there was a baseline low level of experience in the use of the CAM-ICU screening tool, and the incorporation of a video showing how to adequately perform the CAM-ICU was very powerful. Inversely, the lack of daily use of the CAM-ICU screening tool by APPs and fellows (as it is routinely administered by our ICU nurses) may explain the decrease in comfort in its administration among training participants observed at the 3-month follow-up.
There were several strengths of this study. All participants completed the initial survey, postintervention survey, and 3-month follow-up survey. The educational initiative was standard among all groups and could be easily replicated during normal working hours to accommodate both day and night shifts. The training sessions were brief, engaging, convenient, and cost-effective. The APP-led educational initiative helped create homogeneity among all participants, irrespective of their professional categories. The study offered an important approach with practical implications for other institutions as APPs are incorporated in an increasing number into ICU teams across the United States (Moote et al., 2011).
This pilot study has several limitations. The initiative was limited to a single tertiary cancer center, and the number of participants was rather small. We acknowledge there may be limitations to the internal reliability of our quiz by strict survey development criteria, and although beyond the scope of our study, it will be important to formally develop and validate a survey for future interventions. The pilot training program had not been previously tested and data collection was limited to 3 months.

CONCLUSION
Delirium is a major contributor to ICU, hospital, and postdischarge morbidity and mortality. Delirium is also common in hospitalized cancer patients for a variety of reasons. In the critically ill population of patients with cancer, delirium diagnosis and management are rather challenging. Therefore, delirium must be addressed by all oncologic APPs. An APP-led initiative offers a nontraditional but successful and cost-effective training program on delirium screening and management with a focus on cancer patients for multidisciplinary ICU teams.