{'fraction_non_alphanumeric': 0.03016393442622951, 'fraction_numerical': 0.013588342440801458, 'mean_word_length': 5.329490431173622, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Design, implementation and evaluation of a virtual clinical training protocol for midwifery internship in a gynecology course during COVID-19 pandemic: A semi-experimental study

Background Over the past year, the occurrence of COVID-19 pandemic has challenged clinical education for health care students, due to the possibility of exposure to the virus and increased spread of the disease. Clinical training of midwifery students in gynecologic problems, based on the Iran midwifery education curriculum, was also disrupted during this pandemic. Objectives This study was aimed at designing, implementing and evaluating a virtual clinical training protocol for midwifery internship in a Gynecology course. Design A semi-experimental study. Settings Faculty of Nursing and Midwifery, Mazandaran University of Medical Sciences, Sari, Iran. Participants Forty-seven midwifery interns in Gynecology course were recruited during two semesters in 2020. Methods Five steps based on the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model were taken, which included 1) educational and skills needs assessment, 2) design, 3) development via focused group interviews and brainstorming with the presence of the midwifery department members in three sessions, 4) implementation including pretest and posttest, webinar, uploading the information of virtual patients, questions and correct answers, and 5) evaluation including knowledge assessment by a designed questionnaire and skills evaluation by the modified-Mini-CEX checklist. Data were analyzed using mean, standard deviation and paired t-test. Results After training, a significant increase (p < 0.001) was observed in scores of knowledge and interview skills, clinical judgment, consultation, efficiency, professionalism, clinical competence and total score of clinical skills. Conclusions Training for gynecological diseases through virtual clinic promoted knowledge and clinical skills of midwifery interns. To enhance education, a virtual clinic may be used in crisis situations and in combination with teaching under normal circumstances by strengthening the infrastructure and removing barriers.


Introduction
The outlook for education has dramatically changed over the past year due to the COVID-19 pandemic. Higher education institutions have been forced to close universities and turn to online education and digital tools in response to this global emergency. Since the coronavirus crisis continues, it is recommended that universities continuously review and improve their crisis management strategies. The World Health Organization considers e-learning to be a useful tool in meeting the educational needs of health care providers, especially in the developing countries (Al Shorbaji et al., 2015). According to a review study, integrating elearning into medical education is the need of the hour in India and similar countries (Dhir et al., 2017). E-learning promotes professional growth and strengthens the cognitive, emotional and psychomotor aspects of learning (Ruiz et al., 2006). Due to some advantages such as its availability and changing the focus of education from the teacher (traditional education) to students, e-learning is a useful and reliable tool for teachers to provide high-quality education (Ruiz et al., 2006;Dhir et al., 2017). E-learning based on virtual learning is an active and intelligent learning tool that in addition to changing the processes of teaching and learning and information management, it plays a pivotal role in expanding, deepening and stabilizing learning (Isaza-Restrepo et al., 2018;Tabatabai, 2020).
Clinical skills training is a challenging issue in online education (Gormley et al., 2009). As a result of COVID-19 outbreak, universities are facing unprecedented challenges, especially in the field of clinical education. These challenges also apply to the education of midwifery students; thus, alternative opportunities and solutions should be considered. Like medical students, midwifery students are in contact with a large number of patients and clients in medical and health centers. Lack of protective equipment and facilities, little knowledge and skills of students and their large number in wards increase their odds of getting infected and the spread of coronavirus. Therefore, it is necessary to use alternative and complementary methods for face-to-face training to prevent interruption in students' clinical learning. In a study that aimed to teach fetal health assessment to midwifery students, virtual learning was found as effective as traditional education in promoting students' learning (Rabiepoor et al., 2016). Another study compared elearning with traditional learning in health sciences students and found the learning of midwives in the electronic method was significantly higher than other students (Półjanowicz and Latosiewicz, 2010). A study showed that the analysis of clinical cases could promote critical thinking and hypothesizing in physiotherapy students (Soto et al., 2021). Based on the students' experiences, virtual clinic improves some aspects of their clinical skills (Phillips et al., 2013;Rasalam and Bandaranaike, 2020;Rim and Shin, 2021). According to a study in India, nurse midwives students' knowledge and key skills were improved through the virtual classroom training (Agrawal et al., 2016). Some studies in Iran have indicated the need to pay attention to the virtual learning of medical students (Ebadi et al., 2010;ShahsavariIsfahani et al., 2010;Rabiepoor et al., 2016;Seyedi and Yaghoubi, 2020).
Despite significant progress in the implementation of virtual programs in theoretical training, meeting the clinical education needs of students has been neglected. Although scientific evidence highlights the necessity and effectiveness of virtual training programs in upgrading students' skills, studies providing clinical education to midwifery students are limited. According to the Iran midwifery education curriculum, the students must be trained both theoretical and clinical about gynecologic problems. They receive 34 h of theoretical education. Moreover, the final year students are trained in a gynecology clinic for 102 h. Some studies reported the effectiveness of virtual learning in promoting clinical learning, but most of them were based on the students' opinions and experiences (Phillips et al., 2013;Rasalam and Bandaranaike, 2020;Rim and Shin, 2021), so little is known regarding its real impact on the clinical knowledge and skills, especially on midwifery students. On the other hand, education of midwifery students was canceled in the gynecology clinics owing to the COVID-19 pandemic. Therefore, education based on the virtual patients was considered in this study. After identification of midwifery students' educational and skill needs in the field of gynecology, a virtual learning protocol for interns in a gynecologic course was designed, implemented and evaluated in the manner of virtual clinics.

Design and setting
The aim of this semi-experimental study with a pretest-posttest design was to determine the effect of a virtual gynecology clinic training programme on knowledge and clinical skills of midwifery students. This article reports the quantitative part of a mixed methods study which was approved by Mazandaran University of Medical Sciences (code: 7748), and the Ethics Committee of this university (code: IR. MAZUMS.REC.1399.7748) and was carried out at the Faculty of Nursing and Midwifery during two semesters in 2020.

Participants
Forty-seven midwifery students were recruited. All senior students who passed the theoretical and primary clinical gynecology course and selected the internship course in gynecology were included in the study. The exclusion criteria were students who gave up the internship course during the study or did not participate fully in the study stages. After explaining the study objectives and procedure, a written consent was obtained for participation in the study from each student. They were also assured that their names would be kept confidential. Numerous meetings were held with the presence of process facilitators. In these meetings, while reviewing how to implement the protocol with regard to planning, executive activities in the new phase were carefully reviewed, and the necessary planning and coordination were done to implement it.

Protocol of virtual clinic
Based on the ADDIE model, the protocol was implemented in five stages, namely analysis, design, development, implementation and evaluation (Molenda, 2015). This model was used to design former educational courses. It enables course designers to establish a more systematic and effective education. The approach of this model is based on all theories of epistemology, behaviorism and constructivism. This model is called a general pattern since it covers the design of different learning topics, perspectives and environments. This model can be used for any type of learning, both traditional and electronic. Based on the ADDIE model, the following steps were taken.

Stage 1: analysis
At this stage, a meeting was held with the faculty members and senior lecturers in midwifery. Based on their experiences, information was collected about the audience, learning needs, budget and constraints. Assessment of the current situation and analyzing the problem were also done in this meeting. Moreover, the application of e-learning methods to prevent the postponement of midwifery students training in the gynecology course following the COVID-19 pandemic was discussed. The midwifery experts were confirmed the possibility of implementing the virtual training according to the curriculum and educational needs of the students. Then educational needs of midwifery students were identified by examining the level of their knowledge and skills. For this purpose, 10 questions about signs and symptoms, diagnosis and treatment of current gynecological problems were designed by the research team. Moreover, the students were asked about their educational needs through an interview.

Stage 2: design
The main step at this stage was to choose an educational environment in alignment with the goals. The capability of Navid Learning Activity Management System (Navid-LAMS), a web platform in Iran, for implementing virtual clinical education was confirmed. Furthermore, a detailed training plan was prepared, which included the selection of training methods and media and decision on training strategies. After brain storming, the midwifery experts' opinions were discussed and summarized based on the objectives of the gynecology course and the results of needs assessment during three focused group sessions with the research team. Then, a structured model of the virtual clinic protocol was designed and developed. This model included clinical skills in four areas: interview and history taking, problem evaluation, clinical judgment and problem management. For improving these skills, the research team selected 27 common gynecology cases including genital infections, abnormal bleeding, menopause problems, ovarian cysts and abnormal pap smear. A chief complaint was provided for each case as well as 4 questions about 1) what the patient should be asked during the interview, 2) differential diagnosis and diagnosing approaches, 3) main diagnosis and 4) treatment and consultation. Two days were allotted to answering the questions of each step. In addition, it was agreed that before giving new questions in each step, the correct answers of the previous step and more information about each cases would be given. Furthermore, a webinar was designed for improving history taking skill.

Stage 3: development
Selecting and providing the required media and educational materials and deciding on students' activities were performed at this stage. WhatsApp and Adobe Connect were chosen to communicate with the students. It was decided to prepare educational contents in the forms of slides, videos and audio-video files. The activities of students were planned individually and in the form of homework at different stages to improve various areas of clinical gynecological skills and to answer their questions in the different parts of Navid-LAMS. Furthermore, a pretest and a posttest were considered to evaluate the students' knowledge and skills.

Stage 4: implementation
Performing education and supporting the program were the main activities at this stage. It consisted of several steps: a) Pretest: Fourteen multiple choice questions were sent to the students via WhatsApp and 15 min were given to respond. In addition, four genecology cases from 27 designed cases were selected for evaluating the students' skills about interview, clinical judgment, consultation, efficiency, professionalism and clinical competence. b) Educational webinar: The webinar was conducted in one session via Adobe Connect by the research team to teach all students how to take patients' history. Four virtual patients were introduced. Several students from 47 students were randomly selected to respond. The answers were interactively reviewed and discussed with all students. c) Videos on common exams and gynecological problems were uploaded. Moreover, 27 scenarios about gynecological diseases were uploaded in four steps and some questions were asked in each step: (I) Bringing up the problems (improving interview skill): The main problem of the virtual patient was written. The students were requested to ask questions based on the patient's problem to resolve any ambiguities. (II) Problem analysis (improving the problem evaluation skill): The scenarios were written based on the main problem and associated problems. The student was asked to write down the possible diagnoses and diagnostic methods needed to complete the information and arrive at the correct diagnosis. (III) Determining the main cause of the problem (improving clinical judgment skill): The clinical and paraclinical information related to diseases were given. The student was asked to write the definitive diagnosis. (IV) Developing a treatment and consultation plan (improving the problem management skill): The student was asked to write the necessary treatment plan, education, and counseling to the patient based on the definitive diagnosis.
The students were given 2 days to respond to the questions at each step. Furthermore, their answers were checked and the correct answers were uploaded at each stage. d) Evaluation: Evaluation is an ongoing process that starts from the analysis stage and continues throughout the educational design process. At this stage, all aspects of the educational design were assessed (evaluation of the learners' knowledge and performance, media and methods, process of education, revision and correction). The experiences of faculty members and midwifery students were asked through a qualitative study (it was reported in another article). The necessary tests were also run to ensure the acquisition of educational criteria followed by a final evaluation to measure the output of the training course. We used two instruments to collect information at this stage.

Instruments
1) A questionnaire made by the researcher to measure the students' knowledge about patients' assessment, diagnosis, treatment and consultation in common gynecological diseases before and after the implementation of the virtual clinic protocol. Its content validity was confirmed by midwifery experts, and its reliability with Cronbach's alpha was 0.96. 2) Clinical skills assessment checklist to measure students' skills before and after virtual training. Because evaluation was not performed in a real clinical setting, this checklist was prepared with some changes in Mini-Clinical Evaluation Exercise (Mini-CEX). Mini-CEX is a valid and reliable instrument to measure learners' clinical performance (Mortaz Hejri et al., 2020). Because the physical exam skill is not applicable in virtual learning, this criterion was omitted from the Mini-CEX and other criteria were assessed including interview skills, clinical judgment, consultation, efficiency, professionalism and clinical competence. Four virtual cases were selected and some questions related to these criteria were designed for conducting pre and post-test. Three midwifery experts from the research team evaluated the students' answers according to Mini-CEX. A score from a scale of 1 to 9 was given to each criterion, and the scores were classified as unsatisfactory (score:1-3), satisfactory (score: 4-6) and excellent (score: 7-10). Sum of the five criteria scores, which ranged between 6 and 54, was considered as the total score. In this study, to align with other items, the total score was calculated on a scale of 1 to 9. In addition, this checklist measured students' satisfaction with education based on a criterion of 1-9, with a higher score indicating more satisfaction.

Analysis
Data analysis was done by the SPSS version 24. Descriptive statistics including mean and standard deviation and analytical statistics including paired t-test were used. A P-value of less than 0.05 was considered significant.

Results
Totally, 47 midwifery students were assessed. The comparison of their knowledge scores before and after virtual learning (10.0 ± 1.74 and 13.80 ± 1.43, respectively) showed a significant rise with p < 0.001 (Fig. 1). The mean score of students' satisfaction with virtual learning was 8.61 ± 0.53, that was in the excellent range.
The scores of interview skills (history taking), clinical judgment (differential diagnosis and diagnostic assessments), consultation, efficiency (plan), professionalism, clinical competence and total score of clinical skills were significantly increased post-training (Table 1). The pre-training scores of all the criteria, except for professionalism, were satisfactory. The post-training scores improved to the excellent range for clinical judgment, consultation, efficiency, professionalism and clinical competence. Although the score of interview skills increased, it remained in the satisfactory range.

Discussion
To resolve education challenges, especially in special situations such as the COVID-19 pandemic, different e-learning approaches can be used. One of these approaches is virtual patient design. Clinical simulation and standard patient scenarios are reliable instruments in clinical learning (Dutile et al., 2011). Internet-based virtual reality experience provides a safe and non-threatening environment for clinical learning.
Health care providers with designed virtual patient situations can avoid the unnecessary risks of dealing with a real patient (Feng et al., 2013).

Effect of virtual clinic on students' knowledge
The findings of this study showed that virtual clinic increased midwifery students' knowledge about gynecological diseases. In this regard, other researchers reported that e-learning can improve learners' knowledge and abilities via different activities (Feng et al., 2013). In addition, e-learning in medical education facilitates the acquisition of knowledge that is transmitted verbally (Trukhacheva et al., 2011). In one study, clinical e-learning enhanced knowledge through modules made by students (Isaacs et al., 2017). A meta-analysis concluded that the effect of virtual patient on health professionals' knowledge is the same as traditional learning (Kononowicz et al., 2019). Another review concluded that online e-learning was more effective than traditional learning on knowledge (George et al., 2014). However, a meta-analysis study reported the effect of e-learning with designed virtual patients on medicine and nursing students' knowledge was limited compared to traditional learning (Feng et al., 2013). Because in the current study comparison with a control group was not done, it is not possible to draw a definitive conclusion. However, it seems that the effect of situated elearning on knowledge may depend on the tools and methods employed.

Effect of virtual clinic on students' clinical skills
In this study, e-learning based on virtual patient enhanced all aspects of clinical skills of midwifery students in gynecological diseases. A systematic review indicated that online teaching of clinical skills was as effective as traditional medicine (McCutcheon et al., 2015). According to another review, the impact of online e-learning on the skills acquired by undergraduate students in health-related fields was more than traditional learning (George et al., 2014). Some researchers found that the effect of distance learning via multimedia compact disc on nursing students' clinical skills was the same as face-to-face education via lecture (Ebadi et al., 2010).
The virtual patients based learning improved professionalism in midwifery students. A review about the application of e-learning in teaching clinical nursing skills showed this method helped nurses to improve their professional clinical skills due to its flexibility and access to learning content anytime, anywhere, and it is very successful in transferring knowledge to the clinical field (Nezam Abadi et al., 2014). According to a study, virtual patient as a teaching instrument improved physical exam skills (Isaza-Restrepo et al., 2018). Since virtual patients help to improve students' clinical reasoning and decision-making skills (Cook and Triola, 2009), so professionalism may be strengthened. However, a review study reported e-learning enhanced clinical performance in nurses, but not in clinicians (Feng et al., 2013). The discrepancy between the reports may be due to difference in the educational methods and the type of trained skills.
In this study, skill of clinical judgment was enhanced through virtual clinic. Virtual patient is a teaching modality for forming clinical judgment (Montgomery, 2006). Some researchers found that clinical judgment skill of nurses was developed by using virtual stimulation (Fogg et al., 2020;Luo et al., 2021). Critical thinking is a foundation for clinical judgment (Lee and Park, 2019). Furthermore, clinical judgment results from clinical reasoning (Manetti, 2019). Therefore, improving clinical judgment may be due to strengthening of critical thinking and clinical reasoning through virtual patient based-learning (Guise et al., 2012;Isaza-Restrepo et al., 2018).
Clinical competence of midwifery students was improved via virtual clinic in this study., According to a research, virtual patient-based assessment shows expected clinical competence (Forsberg et al., 2011). In study about pharmacy education found that internet-based virtual patients program was useful in the teaching of clinical competency (Jabbur-Lopes et al., 2012). A study also reported virtual patient is a useful and valuable instrument in promoting clinical reasoning of medical students (Isaza-Restrepo et al., 2018). The ability of clinical reasoning is an important factor to achieve clinical competence. Clinical reasoning helps health providers to evaluate patients and their tests to reach the correct diagnosis and come up with appropriate treatment planning (Norman, 2005). E-learning via interaction learning, which  entails encounter of students with multiple cases and creating the proper opportunity to practice with appropriate feedback, has a potential effect on the reinforcement of clinical reasoning (Davids et al., 2015). The virtual clinic improved interview skills of midwifery students. In another study, computer-Consistent with it, some researchers reported virtual patient is a usable tool for developing clinical interviewing skills (Isaza-Restrepo et al., 2018;Washburn et al., 2020). Of course, unlike other skills, interview skills did not reach an excellent level in our study. Moreover, a meta-analysis showed the educational effect of virtual patients was lower for communication skills than for clinical reasoning (Consorti et al., 2012). Because this skill is achieved through face-to-face interaction with patients, virtual learning does not provide this situation. Thus, it seems that learning via virtual patient has a limited effect on interview skills.
In the present study, efficiency was improved by learning through virtual patients. The enhancement of clinical reasoning via virtual patient may have a significant effect on improvement of efficiency (Cook and Triola, 2009). However, the difficulty levels of cases may affect efficiency (Doleck et al., 2016).
The virtual clinic programme in the present study led to promotion of patient consultation skill. This may be related to improving clinical judgment and knowledge about patient needs. According to research, using a virtual patient is effective on consultation skill, especially through improving self-perceived ability (Richardson et al., 2021). Based on the thematic analysis in a qualitative study about interpersonal skills using a virtual patient, adult education was one of the main revealed themes (Bánszki et al., 2018).

Effect of virtual clinic on students' satisfaction
Students' satisfaction with e-learning via virtual clinic was at an excellent level. A review study also reported e-learning leads to nursing students' satisfaction (Nezam Abadi et al., 2014).

Limitations
Lacks of comparison with a control group, cost-benefit assessment of virtual clinic and evaluation of clinical skills in the real environment due to the cessation of traditional clinical education, as well as comparison with blended learning were the limitation of this study.

Conclusion
According to the findings, it seems that e-learning via virtual clinic can help to achieve the educational goals. The weakness of this method lies in the impossibility of conducting clinical examinations and promoting various aspects of professionalism such as patient communication skills. In this regard, it was observed that the level of interview skill was less improved than other clinical skills. Therefore, it seems that this problem can be resolved by integrating this method with face-to face clinical training in the health centers. This method can increase the quality of clinical education not only in critical situations such as the COVID-19 pandemic, but also in normal situations. It is also assumed that this method can be improved by providing variety to clinical scenarios and adding other methods such as training in the skill lab. Comparison virtual clinic trained group with a classically trained group is suggested for more definitive conclusion.

CRediT authorship contribution statement
Conceptualization and methodology: Jila Ganji and Marjan Ahmad Shirvani, Narges Motahari-Tabari, Tahereh Tayebi; Analysis and interpretation of data: Marjan Ahmad Shirvani; Material preparation, data collection and original draft preparation: Narges Motahari-Tabari, Tahereh Tayebi; Review and editing: Marjan Ahmad Shirvani and Jila Ganji. All authors read and approved the final manuscript and have agreed to be accountable for all aspects of the work.
All authors gave final approval to the submitted paper.

Ethical approval
The study was approved by the Ethics Committee of Mazandaran University of Medical Sciences, Iran (code of ethics: IR.MAZUMS. REC.1399.7748).

Funding source
This work was financially supported by Mazandaran University of Medical Sciences (grant number: 7748).

Declaration of competing interest
There is no conflict of interest for the authors.