{'fraction_non_alphanumeric': 0.03859627173777055, 'fraction_numerical': 0.019829851119729763, 'mean_word_length': 4.929896142433234, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessment of Competency of Point-of-Care Ultrasound in Emergency Medicine Residents during Ultrasound Rotation at the Emergency Department

BACKGROUND: Point-of-care ultrasound (POCUS) is the core competency in the Emergency Medicine (EM) residency training. However, there are many methods that can be used to evaluate this competency, and the best practices for teaching ultrasonography to residents have yet to be determined. AIM: The researchers aimed at evaluating the POCUS knowledge and skills of the EM residents after having participated in the POCUS training during their first ultrasound rotation in the Emergency Department. METHODS: A curriculum was developed in the form of a 2-week rotation in the EM residency program at the Department of EM at Khon Kaen University’s Srinagarind Hospital. It consisted of didactic lectures, bedside ultrasound trainings, the journal club, and the process of reviewing the images. Tools were developed, which included a knowledge exam. For each resident, the assessments were administered before and after the rotation. Furthermore, an ultrasound skills test was developed to be used at the end of the 1 year EM residency program. RESULTS: Nine EM residents completed their rotations and the tests. The average pre-training score and posttraining scores were 5.25 ± 1.03 and 8.50 ± 1.20, respectively. The mean difference score between preand posttest was 3.25 ± 1.28. (95% CI −4.321, −2.178). In terms of the ultrasound skills test, the average total score was 26.13 out of 30 (87.1%). Moreover, the residents had higher scores in the aspects of image acquisition (87.5%) and image interpretation (87.5%). However, for the aspect of clinical decision-making, the average score was 75%. The survey questions indicated that with respect to all of the academic activities, the “Bedside ultrasound,” which had encouraged the residents to learn POCUS, was given the highest score (4.75 of 5). CONCLUSIONS: The 2-week ED ultrasound rotation had improved the residents’ EM ultrasound knowledge and skills. Edited by: Sasho Stoleski Citation: Ienghong K, Apiratwarakul K, Tongthummachat N, Suzuki T, Tiamkao S, Bhudhisawasdi V. Assessment of Competency of Pointof-Care Ultrasound in Emergency Medicine Residents During Ultrasound Rotation at the Emergency Department. Open Access Maced J Med Sci. 2021 Apr 20; 9(E):293-297. https://doi.org/10.3889/oamjms.2021.5954


Introduction
Point-of-care ultrasound (POCUS) is one of the core competencies of the Emergency Medicine (EM) residency training. As a result, current graduates from EM residency training programs are now expected to be proficient in bedside sonography. Even though most EM residency training programs do, in fact, teach bedside ultrasound, there is a wide variation in the curricula that are being used [1]. In 2009, The American College of Emergency Physicians (ACEP) introduced competency training in POCUS for EM residents for the purposes of enabling residents to recognize the indications and the contraindications, to acquire the image, and to interpret the image while correlating the information with the patient's clinical features [2]. The Accreditation Council for Graduate Medical Education and the American Board of EM introduced the concept of educational milestones in the context of resident assessment, and 23 POCUS milestones were finalized in October 2012 [3]. Under the ACEP and the Council of EM Residency Directors models, competency is evaluated through standardized checklists and the completion of 150 examinations. However, the number of required examinations in EM is based on the opinions of experts [4], [5], [6].
In the past 15 years, Thai emergency physicians have been introduced to POCUS. However, there are no existing guidelines for teaching POCUS to Thai EM residents. Hence, we implemented the ultrasound rotation for 1 st -year EM residents. In terms of the curriculum for the EM residents, we were able to provide the POCUS training, which included didactic lectures, bedside ultrasound trainings, the journal club, and the process of reviewing the images.
This was the first ultrasound rotation that has been provided for EM residents. In this study, we aimed at evaluating the EM residents' POCUS knowledge and https://oamjms.eu/index.php/mjms/index skills before beginning the rotation and after they had completed their training.

Study design
This was a retrospective, single-center, and observational analytical study conducted in a tertiary university hospital in Thailand. Ethics approval was provided by the Khon Kaen University Ethics Committee for Human Research and was registered with the Thai Clinical Trials Registry (HE631356).

Participants
The 1 st -year EM residents, who were taking part in the ultrasound rotation at the Department of EM at Khon Kaen University's Srinagarind Hospital, were enrolled in this study. No monetary incentives were provided to the participants, and before enrollment, written informed consent was obtained from each of them.

Sample size
All of the 1 st -year EM residents, who studied in the ultrasound rotation during the period from July 2019 to June 2020, were included in this study. The EM residents, who had not participated in this rotation, were excluded from the study. Hence, the total number of participants was determined to be 9 EM residents.

Study protocol
The POCUS training in the ultrasound rotation In this rotation, 1 st -year EM residents participated in the program, which consisted of a 2-week rotation in the emergency department. During this period, bedside ultrasound learning (9 h/week) was provided by a supervisor, who was a POCUS specialist. In addition, there was the journal club (1 time, 3 h/time) and the process of reviewing the ultrasound images (1 time, 3 h/time). There were also the didactic lectures (3 h/week), which consisted of a basic introduction to ultrasound technology (i.e., cardiac, lung, abdomen, inferior vena cava, and aorta), as well as ultrasound protocols, such as the focused assessment with sonography for trauma (FAST) examination, the RUSH protocol, and the CASA protocol.

The ultrasound equipment
The EM residents used the same standard ultrasound machines (the Mindray M9 and Sonosite M Turbo) that are used in the emergency department. In terms of transducers, curvilinear, linear, and phased array probes were provided.

The POCUS training evaluation
At the beginning of the rotation, all of the 1 st -year EM residents were requested to complete the pre-training test, which consisted of 10 POCUS multiple choice questions that presented ultrasound pictures, video clips, and simple clinical scenarios. The individual students, who took the test, were anonymized, and each student was then de-identified and given a code to use. At 2 weeks into this rotation, the anonymized post-course testing (using a de-identified code for each student) was completed by all participants. At the end of the 1 st -year EM residency training, we arranged an ultrasound skills test during which all participants randomly performed the POCUS with real critically ill patients in the EM department. For each patient scenario, the resident received written instructions about what task to perform and to document. The residents were randomly assigned to different patients. However, each of the residents took only 1 exam. The requirement was that the exam had to be completed in 15 min or less. A checklist was used and each exam component was assessed using the Global Rating Scale (1=Poor, 5=Excellent). The ultrasound skills of the participants were evaluated in accordance with six items, which included the aspects of: The recognition of the indication, appropriate of choosing area of scan related patient's clinical, image acquisition, image interpretation, ultrasound knowledge, and clinical decision-making.
At the end of the rotation, we also asked the residents to complete the survey questions. To investigate their perceptions of the events, the participants received an anonymous electronic survey that utilized five-Likert scores. The survey included questions about the learner's opinions about the value of the events in light of his/her own personal development and allowed for open-ended feedback. The de-identified survey responses were then analyzed.

Statistical analysis
The quantitative data were presented as means ± standard deviations, while the qualitative data were presented using proportions and percentages. Differences in the pre-training and post-training tests were compared using a paired sample t-test. A two-tailed p < 0.05 was considered to be statistically significant. All data analyses were performed using Stata version 10 (StataCorp, College Station, TX).

Results
From July 2019 to June 2020, all 9 1 st -year EM residents studied and completed the ultrasound rotation, and all of them completed the test. About 50% (n = 4) of the participants were female.
The average pre-training score and posttraining scores were 5.25 ± 1.03 and 8.50 ± 1.20, respectively. The mean difference score between preand post-test was 3.25 ± 1.28. (95% CI −4.321, −2.178) ( Table 1). In terms of the ultrasound skills test, the patient scenarios included heart failure, chest pain, sepsis, and syncope. The total average score was 26.13 of 30 (87.1%). The minimum total score was 23 (75.16%) and the maximum total score was 29 (96.67%). The residents had the highest score in the aspects of image acquisition (87.5%) and in image interpretation (87.5%). However, in the aspect of clinical decision-making, the average score was 75% (Table 2). The results from the feedback survey, which was administered to the participants, are presented in Table 3. It was found that most participants had used POCUS in the emergency department about 10-15 times/week. In addition, we asked which of the activities could encourage residents to learn POCUS and found that "Bedside ultrasound" had had the highest score (4.75 out of 5). Yet, all of the academic activities, consisting of "Bedside ultrasound," "Journal club," "Didactic lectures," and "Ultrasound image review" had all received scores of greater than 4 points. Findings showed that both the "Utility of the ultrasound rotation to learn POCUS" and the "Utility of learning POCUS in the ultrasound rotation for caring for emergency patients" had been rated at 4.875 points by the participants. However, in regard to their "Confidence to perform POCUS," the participants had given a rating of 3.875 points (Table 3).

Discussion
Over a period of 1 year, 9 1 st -year EM residents were enrolled. This was the first study to investigate the competency of EM residents on the ultrasound rotation in the Emergency department. Our results demonstrated that on the multiple-choice test, the residents' scores had significantly improved. This suggests that the POCUS training, which had been developed for the EM residents, had been successful in improving their POCUS knowledge. It was found that the mean difference score between pre-and posttest was 3.25 ± 1.28 (95% CI −4.321, −2.178), which was consistent with the previous studies [7], [8], [9]. For example, a study by Noble VE noted an increase in ultrasound knowledge scores among EM residents participating in Emergency Ultrasound rotations. Furthermore, these results were consistent with a study conducted by Costantino et al., who showed that higher test scores had been associated with residency programs, which had ultrasound rotations (odds ratio 1.82; 95% confidence 95% CI 1.29-2.55) [10]. In terms of the numbers of hours for the ultrasound rotation, some recommendations for training EM residents had ranged from 4 h to 2 weeks of training [11], [12]. Current ACEP guidelines suggest an equivalent of at least 16 h of didactics and a minimum of 25 ultrasound scans for each type of scan performed [2]. These were the variables found among the ultrasound curricula. However, our study, which consisted of a 2-week rotation, had clearly demonstrated improvements in ultrasound knowledge among the EM residents.
In terms of the evaluation of the ultrasound skills, our results showed that the residents had maintained good ultrasound skills even after 1 year had passed. The residents performed well in the aspects of image acquisition and image interpretation, which was consistent with results from a study by Schmidt et al. [5], in which it was found that residents had performed well on image acquisition with an average score of 85.7% for core skills, which included FAST, aorta, echocardiogram (ECHO), pelvis, and for central line placement. Moreover, for the advanced skills (ovaries, advanced ECHO, and advanced aorta), the residents had scored 74%. Findings from a study by Smalley et al. indicated that having a dedicated ultrasound rotation and spending a number of weeks rotating in ED had had a statistically significant effect on image acquisition [13]. In this study, we also evaluated the residents' clinical decision-making processes by giving them opportunities to apply the knowledge that they had learned with real patients in real clinical scenarios. For the assessment of the clinical skills, this method has also been used to good effect in many studies [14], [15], [16]. Nevertheless, some barriers were experienced because this form of evaluation requires the presence of attending physicians, who have more experience in incorporating POCUS into clinical encounters. Furthermore, the participants were evaluated by utilizing four scenarios, which only cover some parts of ultrasound knowledge.
Based on the survey questions, of all of the academic activities that could encourage the participants to learn POCUS, "Bedside ultrasound" had shown the highest score. This was consistent with findings from a study by Favot et al., when the participants in this study were asked how the curriculum could be improved; it was found that a majority of the students noted that they had valued the "one-on-one instruction" with the bedside ultrasound (55.2%) [17]. Not only had the feedback revealed high scores for the "Utility of the ultrasound rotation for learning POCUS" but it also indicated that the participants had preferred using POCUS for clinical settings to aid in diagnosis and to better the care of their patients after completing the POCUS training.
This study is limited by its small sample size. In addition, because this study was conducted at a single institution, the results from this study may not be generalizable to other residency programs. Findings from this study indicated that after 1 year of regularly using POCUS in their everyday clinical practices, most of the EM residents had been able to maintain their ultrasound skills. However, the following were not controlled: (1) The number of scans that the participants performed, (2) the random variability in the proportion of each type of scan performed by each participant, or (3) any ultrasound experience that the participants may have had following the ultrasound rotation. Therefore, any association between the number of scans, personal ultrasound experiences after the ultrasound rotation, and the scores of the ultrasound skill test could not be demonstrated [18], [19], [20], [21], [22].

Conclusion
This study found that as a result of completing a 2-week ultrasound rotation, the EM residents had been able to effectively learn POCUS in the areas of ultrasound knowledge and ultrasound skills, especially with regard to image acquisition and image interpretation. Our research showed that EM residents had preferred the academic activity of "Bedside ultrasound" in the ultrasound rotation, which had encouraged them to learn POCUS. For future studies, evaluation of the trainees should be carried out across a broader spectrum of case scenarios and should be validated based on the varying levels of experience that exist among the trainees. The surgeon-performed ultrasound: A curriculum to improve