{'fraction_non_alphanumeric': 0.024731291485562183, 'fraction_numerical': 0.011792850163415209, 'mean_word_length': 5.176899063475546, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improving Staff Knowledge and Attitudes Toward Providing Psychosocial Support to NICU Parents Through an Online Education Course

Supplemental Digital Content is Available in the Text.


Psychosocial Support to NICU Parents Through an Online Education Course
The lack of communication skills in sensitive situations increases staff's own stress and contributes to on-the-job burnout. 12 Communication shortfalls also present challenges in the provision of familyintegrated care, in which NICU staff form more egalitarian partnerships with parents and encourage greater involvement at their infant's bedside. Making a paradigm shift, in which unit culture will "begin and continue to develop and evolve to become one in which families feel like valued and supported members of the care team," 13 calls for new skill development among multidisciplinary staff 14 as well as interprofessional work.
Numerous interventions have been designed to improve communication skills of nurses 15 and physicians 5 , 16-21 with families of NICU infants. Topics include the delivery of bad news, end-of-life issues, and discussing options for infants to be born at the edge of viability. While some communication interventions include intensive simulation training, 5 , 16-19 others offer specific, targeted protocols for how to improve parent-provider communication. 8 , 21-29 Several have been developed for improving nurses' skills in communicating on the topics of palliative and bereavement care, some not specific to the NICU environment, 30 , 31 while others are focused on the NICU. 20 , 32-34 These programs include combinations of live lectures, interactive workshops using case studies, audiovisual examples, and videotaped roleplays. An educational program for neonatal fellows focused more on acquainting them with patient qualifications for comfort care and withdrawal of support. 20 To our knowledge, there is no comprehensive educational program for NICU staff to help them both understand and meet the emotional needs of NICU parents and families across all aspects of the NICU experience.
In 2015, an interdisciplinary group, convened by the National Perinatal Association (NPA), published the "Interdisciplinary Recommendations for Psychosocial Support of NICU Parents," 35 which highlighted the need for NICU staff education on a variety of topics. 4 The NPA has since partnered with Patient + Family Care and the Preemie Parent Alliance, to form My NICU Network ( www.mynicu network.com ), and to develop an education course for multidisciplinary NICU providers with course content mirroring the NPA recommendations. The goal of the course, "Caring for Babies and Their Families," is to improve staff confidence and skill in their ability to care for families in crisis, and in turn improve the patient and family experience.

SPECIFIC AIM
The aim of this pilot study was to determine whether NICU staff perceived improved knowledge and attitudes of their ability to provide comprehensive

What This Study Adds
• This study highlights a new online education course that was shown to be effective in improving knowledge and attitudes about providing psychosocial support to neonatal intensive care unit (NICU) parents among NICU nurses; significant improvement was sustained in follow-up 6 months later. • The study confirmed that NICU staff understood that NICU parents need emotional support, but in general staff did not feel they had adequate communication skills, strategies, and confidence to provide this support. • The study identified 2 groups, night shift staff and staff with fewer years of service, whose lower pretest scores in several areas demonstrated that they may have increased needs for educational opportunities. • Education in palliative and bereavement care was identified as a high priority on pretest, posttest, and 6-month follow-up surveys.
psychosocial support to NICU parents, using a selfassessment measure, both before and after taking an online education course and at 6 months' follow-up.

DESIGN
After course content was created by the My NICU Network team, an informal review of the relevance and appropriateness of the course's content took place among 12 NICU staff who were recruited from the My NICU Network Facebook page. Modifications to the content were made in response to feedback. Next, a total of 350 staff at 2 hospital NICUs were invited to participate in the time series pre/posttest/6-month follow-up descriptive study outlined here.

INTERVENTION The Course
The course was developed by an interprofessional group, which included a neonatologist, a neonatal nurse, 2 NICU psychologists, an occupational Hall et al therapist, and a volunteer group of 12 NICU graduate parents. The 7 learning modules, each providing 1 hour of continuing education credit, are evidencebased, story-driven, clinically relevant, traumainformed, and resource-rich. Content includes written narrative outlining the evidence base for recommended practice changes; audio clips and illustrative quotes provided by NICU parents; video clips, some demonstrating simulated conversations between NICU staff and parents (developed by staff at Children's Hospital of Philadelphia); links to other Web sites to encourage exploration of related resources; and numerous curated resources to download. The goal of having an entire NICU staff take the course concurrently is to accelerate systems change and provide collegial support, as the culture transforms to all staff understanding that psychosocial support of parents is an integral part of their professional role. The modules in the course are based on NPA's recommendations in the following topic areas: (1) communication skills, 4 (2) providing emotional support to parents, 36 (3) peer-to-peer support, 37 (4) family-centered developmental care, 38 (5) palliative and bereavement care, 39 (6) discharge and follow-up support, 40 and (7) supporting staff as they support families. 4

Instrument
The survey instrument was developed by a multidisciplinary team of NICU experts, including a neonatologist, 2 psychologists, a nurse, and an occupational therapist. The instrument measured participants' subjective assessments of their knowledge and attitudes toward supporting NICU parents by indicating their agreement with 33 statements using a Likert scale of 1 to 6 (1 = strongly disagree to 6 = strongly agree), see Supplemental Appendix 1 (available at: http://links.lww.com/ANC/A46 ).
As no similar surveys were available for comparison, validation of the survey instrument was derived from the opinions of experts who created it. Reliability tests were conducted on the survey and within each module. The Cronbach α for the pretest was 0.952, 0.813 for the posttest, and 0.94 for the 6-month follow-up. All modules had a Cronbach α above 0.7 except for all instances of peer-to-peer support; because of this, results relating to peer support will be displayed, but not emphasized in results or discussion.
The survey was administered before taking the course (pretest); this version also included basic participant demographic questions. Participants then had 3 months to take the course, and completed a module-specific posttest after finishing each. Posttests were identical to pretests, excluding the demographic questions. Brief, weekly e-mails were then sent to participants over the 6 months following course completion, reminding them of key points from course content. At the end of this period, they were asked to take the same survey to assess retention. All responses were de-identified.
Free continuing education credits were awarded as an incentive for participation. Nurses who completed all modules received 8 credits, and physicians and social workers received 7; those completing less than 7 modules received the commensurate number of credits. The course has not been certified to provide credits to other ancillary professionals.

Statistical Analysis
Descriptive and inferential statistics were completed using SPSS v25 (SPSS Inc, Chicago, Illinois). Categorical variables were compared using analysis of variance tests; continuous variables were compared using independent samples t tests. Identification of paired data sets for scores at all time points was not possible due to computer issues; thus, scores were pooled to compare as independent samples. Both pre-/post-and pre-/follow-up P values were calculated for each survey item, and P values as well as effect sizes (Cohen's d statistic 41 ) were calculated for module subscores.

Procedures (Institutional Review Board)
Upon institutional review board review at both institutions, this project was granted exempt status, as it was determined not to be research involving human subjects.

Demographics
A total of 114 NICU staff registered for the course; 76% (87/114) started and 58% (66/114) completed the course. There were notable differences in the rates of course completion between the 2 hospitals (78%, n = 31 at NICU 1, vs 47%, n = 35 at NICU 2). The majority of participants (88%, 73/83) were nurses. Demographics of participants for whom demographic data were available are shown in Table 1 .

Pre/Posttest and 6-Month Follow-up Data Comparisons
Pre/posttest and 6-month follow-up mean scores on all survey items, and standard deviations, are shown in Supplemental Table 1 (available at: http://links.lww.com/ANC/A47 ) with module subscores summarized. All survey items had higher posttest mean scores than pretest scores, and all module subscores were significantly improved from pre-to posttest at the P < .001 level, demonstrating participants' improvements in knowledge and attitudes immediately after completing a module. On the posttest, using Sawilowsky's criteria 42 for categorization of effect size, the emotional support module showed a very large effect; palliative care, communication, staff support, family-centered developmental care Psychosocial Support to NICU Parents Through an Online Education Course however, significance in staff support module was lower at P < .05 and effect size was small.
Pretest scores were highest in the family-centered developmental care module. Participants still demonstrated statistically significant improvements in this module at both periods, and continued positive effect sizes (large and medium, at posttest and 6-month follow-up, respectively). The next highest pretest means were found in the discharge and follow-up module; again, significant improvements were demonstrated at both periods, although effect sizes were medium and small, at posttest and 6-month follow-up.
In general, participants scored higher on pretest survey items indicating that they knew the importance of providing psychosocial support to parents and/or engaging in self-care, than they did on items indicating that they had strategies for such and/or were comfortable doing so (see Table 2 ).
Pretest mean scores on 4 of the survey items were significantly higher among participants on day shift compared with night shift (see Table 3 ). There were no differences in posttest or 6-month follow-up mean scores related to shift assignment. Four pretest survey items having to do with emotional support and palliative and bereavement care were significantly higher among participants with more years of experience in the NICU compared with those with less experience (see Table 4 ). While, in general, scores were higher on these items among staff with increasing years of experience, on 2 of the 4 items, scores were slightly lower for staff with more than 20 years of experience after peaking among those with 11 to 20 years of experience. There were no significant differences in posttest or 6-month followup mean scores depending on NICU experience. Figure 1 shows responses to the immediate postcourse evaluation questions in graphic form using data from all available participants.
A 2-question follow-up survey sent to the 48 noncompleters found that the major barrier to course completion was lack of time (93.3%, 14/15). The average rate of weekly follow-up e-mail openings was 26%.

DISCUSSION
This pilot study evaluated an online NICU staff education course as an intervention to increase staff knowledge and attitudes toward providing comprehensive psychosocial support to parents. The majority of participants were nurses, reflecting their preponderance on staff. Approximately one-fourth of those who registered for the course never started the course. Some possible explanations include lack of incentives and/or perceived lack of time to take the course. However, once participants became engaged in the course, most finished. This finding emphasizes showed large effects, and discharge showed a medium effect (see Supplemental Table 1, available at: http://links.lww.com/ANC/A47 ).
At 6-month follow-up, excluding the 3 questions on peer support, nearly all (28/30, 93%) mean scores on individual survey items continued to be greater than pretest means, although most (25/30, 83%) were lower than posttest means. Statistically significant improvement was maintained at the 6-month level on all module subscores compared with pretest values. Learning effect sizes at 6-month follow-up either stayed in the same range as on the posttest (communication) or dropped 1 level (all others), with staff support and discharge care dropping into the small effect size, although these still met the average effect size for learning interventions, which is 0.4. 43 Participants had the lowest pretest scores in the categories of peer support, and palliative and bereavement care. The significance of improvements in scores and in learning effect sizes demonstrated in peer support cannot be considered reliable due to the lack of internal consistency of the survey in this area. Posttest and 6-month follow-up scores in the palliative care module, although significantly improved, remained the lowest (excluding peer support) of all the learning modules.
The modules on communication, providing emotional support, and staff support displayed pretest means in the middle range. Both the communication and emotional support modules maintained statistically significant improvement at the P < .001 level at 6-month follow-up, as well as large effect sizes;  (51) 37 (44) 4 (5) Abbreviation: NICU, neonatal intensive care unit.
the necessity of developing a strategic implementation plan in each NICU offering the course to encourage maximal staff participation. NICU staff showed improvement in both knowledge and attitudes after taking part or all of the course, and significant improvement was maintained at the 6-month follow-up mark. Learning effect sizes for the various modules ranged from very large to medium on the posttest. At 6-month follow-up, all learning modules maintained effect sizes at or above the average effect size for learning interventions. 43 The module in which staff indicated the greatest deficits in knowledge and communication on the pretest was in peer support. Participants indicated understanding the importance of providing peer support to parents, but did not ensure that they referred parents to peer support programs. Neither NICU involved in the study has its own peer support program, rendering the number of responses on the related survey questions small and lowering internal consistency of the questions. Although the American Academy of Pediatrics (AAP) recommends that all families of pediatric patients be offered peer support during a hospitalization, 44 few NICUs have established their own peer support programs. This draws attention to the need to educate professional NICU staff about the importance of developing local peer support programs, 37 and about resources that are already available for NICU parents on the Internet, to which parents

Psychosocial Support to NICU Parents Through an Online Education Course
can be referred. The Preemie Parent Alliance maintains a member directory, which lists more than 30 active peer support organizations nationally. 45 Pretest scores on items relating to palliative and bereavement care were among the lowest, as were the posttest and follow-up scores, indicating that although staff showed large and medium learning effects, respectively, this is an area where staff continue to be in greatest need of both education and support. This also aligns with previous findings about staff's desire for increased education in this area, 5 , 7 , 32 which might include practice with simulated conversations, reinforcement of learning points, and formalized mentoring programs.
Pretest responses in the communication category showed that staff knew the importance of communicating and felt they have basic skills (listening, communicating bad news, communicating without judging), but lacked confidence in the most difficult of situations (comforting distressed parents when busy taking care of their sick children). After completing this module, staff indicated an increased understanding of communicating with parents, as well as more comfort doing so. The large learning effect size at both followup points reinforces the utility of this module.
While staff indicated knowledge of the importance of self-care with a relatively high score on the pretest, the lowest pretest score in the category of staff support was in response to having specific strategies to help coworkers deal with burnout. Burnout rates among staff in NICUs have ranged from 7.5% to 54.4%, 46 , 47 making it imperative that NICU staff develop and implement strategies to help them deal with burnout. While posttest and 6-month follow-up scores on this item showed improved knowledge of strategies, the learning effect size for the module fell from a large to a small effect size at 6-month follow-up, indicating the need for further emphasis on this topic until it is more fully incorporated into NICU culture.
Most staff agreed on the pretest that they understood the importance of providing parents with

FIGURE 1
Participants' answers to the postcourse evaluation questions in graphic form. Hall et al emotional and psychosocial support to reduce parental stress during the NICU experience. However, pretest scores reflecting whether staff had examples or strategies in these areas were lower, as were scores reflecting their confidence in their abilities to provide this support. The emotional support module showed very large and large learning effect sizes at the posttest and 6-month follow-up periods, respectively, revealing that participants gained both strategies and confidence in providing support to parents. Pretest scores in the discharge category were among the highest, showing that staff generally felt competent in this area; nonetheless, gains were still made on all items at posttest and 6-month followup, although the learning effect size was small at 6-month follow-up.
The category of family-centered developmental care had the highest mean pretest scores of any module, suggesting that initiatives on providing family-centered care have successfully reached NICU nurses at the bedside. The importance of encouraging families to become involved in the care of their infants has been emphasized by many, 1 , 38 , 44 , 48-50 and has been a major tenet of neonatal care since the early 1990s. 51 However, implementation across NICUs nationally has been variable 50 likely due to factors such as staff awareness, financial considerations, and/or resource availability, leaving continued room for improvement. 38 , 48 , 49

Results by Shift
The pretest gaps found between night and day staff in terms of their knowledge and confidence on several survey items (discharge planning, palliative and bereavement care, and providing comfort to parents) were no longer present on the posttest. It is not surprising that night shift workers would not feel as prepared to help parents with discharge, since most discharges occur on the day shift; but in many NICUs, discharge teaching increasingly takes place on both shifts. Both shifts must be aware of and be ready to assist with tasks needing completion prior to discharge, especially given the needs for cross-coverage, increased discharge pressure related to ensuring bed availability, and continual drive to improve familycentered care. According to the AAP, careful preparation for discharge may reduce the likelihood of hospital readmission and/or death during the first year after the birth. 52 Therefore, both day and night staff must be informed regarding best practices so that they can present clear and consistent information to families.

Results by Years of Experience
Participation in the course reduced the pretest gap between less and more experienced staff in terms of their knowledge and confidence on items related to providing emotional support and palliative and bereavement care. It is not surprising that nurses with less experience have lower confidence in their skills in the area of palliative and bereavement care, as they may not have had the experience of caring for as many infants who did not survive compared with nurses who have been practicing longer. In addition, nurses with over 20 years of experience may have completed their education prior to the relatively recent introduction of palliative care programs into NICUs in the late 1990s. 53 , 54 The AAP published recommendations about the importance of palliative care for children in 2000, 55 addressed noninitiation or withdrawal of intensive care among high-risk newborns in 2007, 56 and then outlined how to provide bereavement care to parents of pediatric patients in 2012. 57 The National Association of Neonatal Nurses similarly released a position statement on palliative care in 2015. 58 Palliative care programs continue to be developed and their utilization expanded in NICUs nationally. [59][60][61][62][63] Training programs aimed at improving providers' skills and comfort with palliative care have become more readily available (eg, Resolve Through Sharing and End-of-Life Nursing Education Consortium) 31 , 34 to help reduce this educational gap.

Postcourse Questions
The large majority of participants endorsed the course's value in increasing their knowledge and changing practice, and would recommend the course to colleagues.

6-Month Follow-up Data
Although the response rate from follow-up surveys was low, the data showed that participants sustained improvements in their knowledge and attitudes about supporting parents, although not always to the immediate posttest survey level. Since the rate for opening weekly follow-up email was low, it is unclear whether this affected retention of learning. While some communication training programs for healthcare workers have not provided follow-up results, 30 , 64 others have looked at training outcomes ranging from 1 16 to 12 65 months at follow-up, showing varying degrees of sustainability of learning and integration of skills into practice. However, selfrated ability and satisfaction of participants completing communication skills training courses do not always correlate with objective measures of performance. 64 Communication training programs need to demonstrate both sustainability and performance improvement; this might best be enhanced by incorporating scenario-based teaching. 66

Recommendations for Practice
Several topic-specific recommendations are made based on outcomes of this study. First, each NICU should evaluate how they can continue to refine and improve family-centered developmental care and discharge planning and follow-up support based on best practices. Second, education on palliative and Psychosocial Support to NICU Parents Through an Online Education Course bereavement care should be offered to NICU staff on an ongoing basis, especially around communicating with and supporting parents during times of loss. Third, NICUs should work toward creating their own peer support programs, and until they have one, staff need to be made more aware of peer support resources in their community, regionally, and nationally in order to offer them to parents. Next, staff with fewer years of experience are most in need of education in the area of providing psychosocial support, especially around situations involving palliative and bereavement care. Nurses with less experience could likely benefit from a mentor nurse during challenging situations with families. Last, but with equal importance, night shift staff need equal and adequate exposure to staff education so that all are prepared to support parents to the same degree.

Limitations
There are a few limitations to this study. Pooled mean data from the surveys were presented instead of matched pre-/post-/follow-up test pairs due to the limited number of identifiable pairs, resulting from participants' use of different computers at different time points to complete the surveys. The relatively small participant number limits our ability to generalize results to all NICU nurses, and the sample homogeneity limits our ability to generalize to NICU providers other than nurses. Furthermore, the small number of responses on one of the questions related to peer support gave a low value for internal consistency of the survey in this area, rendering results in this domain unreliable.
The primary barrier participants experienced in completing the course was lack of time. Nursing staff at NICU 1 were eligible for paid time to take the course; this may have contributed to their higher rates of starting and completing the course compared with staff at NICU 2. Another factor may relate to NICU size: NICU 1 is much smaller and has lower patient acuity than NICU 2, leaving staff more time to complete the course. It is also possible that the 3-month period was not long enough for some staff to complete the course. Furthermore, staff time constraints may indicate that course modules should be shortened.
The relatively low numbers of participants who completed the 6-month follow-up survey could reflect either staff time constraints or self-selection (ie, that those who were most positive about the course from the beginning were most likely to retake the follow-up survey).
Having a well-designed implementation plan specific to launching this course will encourage maximal participation and improved opportunity for course completion. Other considerations include incentivizing staff for their time to take the course, perhaps through employee financial support, or otherwise making course modules part of the curriculum in ongoing staff development meetings, incorporating the course into broader quality improvement initiatives, or requiring course completion as part of competency requirements.

Implications for Future Research
Future research should address the main barriers to an entire NICU staff taking the course: providing relief time and incentivizing staff to take the course. Other strategies to sustain learning in follow-up should be explored. The generalizability of our findings to a larger and more diverse group of NICU staff, including all disciplines, should be determined. A next step should be to determine whether improved knowledge and attitudes lead to behavioral change at the bedside. Including families in outcome measures would offer another critical lens through which to evaluate the ultimate success of the course.

CONCLUSIONS
In summary, results demonstrate that the online staff education course "Caring for Babies and Their Families" was highly effective in improving NICU staff's knowledge and attitudes toward providing psychosocial support across a broad range of categories, as well as in providing them with strategies and confidence. Significant improvement was shown in all module subscores at both the posttest and 6-month follow-up periods. Areas with the greatest participant knowledge deficits were identified and can be targeted for future interventions. The majority of staff agreed or strongly agreed that taking the course improved their knowledge and would change their practice, and that they would recommend the course to their peers.

Summary of Recommendations for Practice and Research
What we know: • Provider-parent communication is a critical determinant of how NICU parents cope with their situation and how satisfied they are with the care of their infant.
• Parents are not always satisfied with the communication and support they receive.
• Many neonatologists and neonatal nurses do not feel they have adequate skills to support distressed or anxious parents.
• New paradigms of care such as family-integrated care require involving family members on the care team; providers need new knowledge and skills to optimize family involvement.
What needs to be studied: • Methods of educating NICU staff (including nurses, physicians, and ancillary staff) about the need for psychosocial support among NICU parents.
• Methods of educating NICU staff about how to provide comprehensive psychosocial support to NICU parents.
• Methods of educating NICU staff about how to support each other and themselves, to prevent burnout.
What can we do today that would guide caregivers in the practice setting considering use of this evidence for guiding practice: • NICU administrators should ensure that caregivers get adequate exposure to continuing education in skills related to the provision of emotional support to parents and families.
• In particular, night shift staff and staff with fewer years of experience should be encouraged to participate in educational offerings.
• Education in techniques of providing palliative and bereavement care should be a high priority, as caregivers indicate their need for this kind of training.