{'fraction_non_alphanumeric': 0.022872713760786093, 'fraction_numerical': 0.005243383840469015, 'mean_word_length': 5.250838709677419, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Effect of Interactive and Effective Lecturing Workshop for Developing Faculty Members in Teaching: An Experiment of Utilizing Peer Observation of Teaching and Feedback

Background: Whereas much has been written about the strategies, barriers and facilitator factors of effective and interactive lecturing in medical education little has been written about the effective and interactive lecturing skills educational programs for medical teachers based on peer observation of teaching. Objectives: The current study aimed at designing and implementing an interactive and effective lecturing workshop using peer observation and feedback, and finally evaluating its results. Methods: The current descriptive study was conducted in Tehran University of Medical Sciences from 2015 to 2016. The study population consisted of faculty members participating in the effective and interactive lecturing workshop and the study subjects were selected by convenience sampling method. The Kirkpatrick method was used to assess the workshop; for this purpose, the level of reaction, learning, and performance were evaluated using a valid and reliable questionnaire, as well as the one minute note technique, and a form addressing the extent of using interactive techniques in the classroom six months after participation in the workshop. Data were analyzed by using the SPSS 22. Data are presented as frequency and mean where appropriated. The notes were analyzed using manual content analyses. Results: The participants believed that the workshop could successfully encourage them to use lecturing principles and interactive lecturing techniques, and provide them with the opportunity to practice and rethink the teaching process. The interactive techniques mostly used six months after participation in the workshop belonged to the question and answer (Q& A) techniques, active evaluation, and use of scenario. Conclusions: It seems that the provision of training opportunities, observation of performance, and giving feedback were effective to improve the quality of empowerment programs. It was suggested that other empowerment programs should also address this point.


Background
In recent years, there are a lot of changes in the revision of medical science curricula and shift toward the integration of basic and clinical sciences, but nonetheless, lecturing is still at the forefront of other teaching methods. Because of the bad delivery of lectures, unfortunately they are considered as a 'traditional' form of teaching which are seen without structure that leads passive learning with the newer learning approaches. These problems are not actually related to the nature of a lecture, but because of incorrect use of it and failure to observe its principles (1,2), since lecturing is the most commonly used teaching method at medical universities (1)(2)(3). Although this method is one of the oldest and most widely used methods in medical education, unfortunately today educational experts believe that this useful educational method is not properly applied (3,4). Problems with teaching methods are among the most important problems always addressed in various evaluations and student surveys; to such an extent that after reviewing most of the curriculum, the dominant teaching method in basic sciences is the interactive lecturing method, which is run with a lot of bugs. lack of familiarity of professors and the inadequacy or inefficacy of the current methods to empower the faculty members. In many texts, the problems related to proper empowerment of faculty members, lack of motivation, and inadequate attitude toward the employment of various applicable teaching methods are among the most important reasons for not using them (2,4). In a recent study conducted at Tehran University of Medical School, one of the most important indicators from the viewpoint of faculty members was the problems associated with their empowerment and inadequacy and inappropriateness of available teaching methods (5). On the other hand, most professors often criticize the manners in which the workshops are held, since they do not give them enough opportunity to practice and criticize the performance. After observing these problems, the authors gradually reviewed the existing studies and discussed the way in which the lecturing workshop is organized. To design the workshop, the available literature and the experiences of successful universities were also used (6).
The review of similar studies showed that in various development programs, the improvement of quality of the lecturing method is emphasized. In such programs, a particular attention is also paid to the use of observation, feedback, and rethinking stimulation methods (1). Nasmith and Steinert designed a four-hour workshop to teach interactive lecturing to the faculty members of McGill School of Medicine. The designed program comprised of barriers to interactive lecturing, introduction of interactive lecturing strategies, and providing an opportunity to practice such strategies during the workshop. The results of their evaluation showed that most of the participants considered the workshop useful and after six months they often used interactive lecturing methods in their teaching practice (1).
In a study by Mcleod et al. on the effect of peer observation and feedback using the recorded lectures, it was observed that this method was highly acceptable among participants and led to improved learning. They believed that lecturing skills could be improved by the peer observation and feedback (7). Pattison et al. conducted another study on the empowerment of medical professors teaching skills using peer observation and feedback method in McMaster University. Participants rated the method very useful to enhance their teaching abilities. They also believed that the provided feedback led to a rethinking of their performance (8).
From the totality of studies conducted in this regard it is perceived that the empowerment programs can be effective to expand the employment of active and principled learning methods, but poor empowerment programs that do not provide faculty members with opportunities for practice and feedback cannot provide proper chance of experience for them to practice what they learn (9). Many studies are focused on lecturing, comparing it with active teaching methods and barriers, and facilitators (1), but limited studies are conducted on empowerment programs and describing the experiences gained from peer observation and feedback.

Objectives
The current study aimed at describing the experience of Tehran University of Medical Sciences in using the peer observation of teaching and feedback methods to empower faculty members in effective and interactive lecturing, determining their satisfaction with interactive and effective lecturing workshop, and evaluating the outcomes of using interactive lecture techniques.

Methods
The current descriptive, cross sectional study was conducted at Tehran University of Medical Sciences from 2015 to 2016. The study was designed, implemented, and evaluated in three stages. The study population consisted of faculty members participating in an effective and interactive lecturing workshop. Subjects were selected using the convenience sampling method.

Design
In order to design an effective and interactive lecturing workshop, a group of faculty members and qualified students interested in lecturing was formed as the scientific and executive committee of the program. This group included the Director of the Research and Development Center of university, two faculty members interested in teaching with strong proficiency in lecturing, oratorical skills, teaching experience, and one PhD student in medical education with a history of teaching-learning activity and empowering the faculty members. Three two-hour meetings were held to design the workshop. The result of the meetings was the formulation of principles, objectives, method of implementation, and the establishment of the evaluation method.
The workshop should provide an opportunity for professors to give feedback and effectively and interactively practice lecturing.
The workshop should also provide the opportunity of reflection, contemplation, and self-assessment for professors. The program should emphasize on the application of the principles of effective and interactive lecturing, and not only the presentation of theoretical content. 2 Strides Dev Med Educ. 2018; 15(1):e86954.

Implementation
The program was implemented in three courses as a three-day 20-hour workshop. The first day of the workshop, entitled "effective lecturing", was held with a review of videos of lectures by professors recorded before the workshop. After reviewing the texts related to the lecturing challenges and its benefits, Gagne's nine principles were introduced to streamline the structure of the lecture, and finally, the rhetoric and body language principles were expressed. Subsequently, participants in small groups practiced how to compose a lecture and each of them was given a 10-minute opportunity to present their lectures in order to apply the principles taught at the workshop. During the presentation, all lectures were video recorded by the participants, and then were screened to receive peer feedback, and each of the subjects received feedback in terms of rhetoric skills and lecturing principles. A pre-designed checklist was used to provide feedback. The observation and feedback process is presented in Figure 1.
On the second day of the workshop entitled "interactive lecturing", the adult learning principles were first introduced. Then, 10 interactive lecturing techniques, including "working in small groups, Q& A, quiz, short movies, debating, flashcards, role playing, handouts, scenarios, and learning evaluation", were actively introduced with emphasis on their application in the classroom. Then, subjects in five groups of seven participated in the practice of designing an interactive lecture using such techniques; they presented their lecture within 10 minutes, and then received its feedback.
The third day, entitled "problem students and using software in presentation", began with an engaging lecture on problematic students in the classroom. Then the techniques to manage such problems were expressed. The discussion was managed by the question and answer (Q& A) method. Then, the principles of using Prezi and the slide set were put into practice. At the end, in the small groups, the professors examined their own as well as their peers set slides and provided feedback using a checklist.
Hence, two checklists were used in the workshop to provide feedback. The first checklist was designed to provide feedback to the effective lecturing and the second checklist, to feedback on the slide set. In order to design these checklists, the texts and comments of the scientific committee of the workshop were reviewed.
Checklist of feedback comprised of the principles of lecturing including sound management (tone and volume of voice), speed of speech, making pauses in speech, correct pronunciations, speaking like a common conversation, not formally like reading a book, using the names of the audience, maintain and keep eye contact, cheerful face, the proper use of hand movements, walking all sides of the classroom energetically, well-dressing, smiling at the right time, listening actively to the answers, looking at the audience, not at the screen, presenting the slide contents in brief and the self-language, not just reading the slides. The feedback was given based on the following scale: "He is adequately qualified to perform such skills and does not require additional training; he can perform this skill, but requires more training; he cannot perform this skill and requires much training; the skill is not applicable to present this material and no skill application was observed during the presentation".
The slide set feedback checklist included two parts as contents and template. In the content part, seven, and in the template part, 13 evaluation criteria were included as follows: "The content used is up-to-date; the contents of the slides are scientific and based on recent knowledge; important and applicable information is provided; images, figures, and diagrams are correlated with the content of each slide; figures and diagrams transfer the message perfectly; the reference of each slide is specified if necessary; the order of contents presentation is appropriate in slides (introduction, methodology, results, discussion, and conclusion); it is easy to read the content of the slides; the type of font used is appropriate; the font size used is appropriate (class above 200 students: heading = 42 and body = 36; class under 200 students: heading = 36 and body = 28; class under 50 students: heading = 32 and body = 24); type and size of the font are similar in all headings; type and size of the font are similar in all texts; background and font colors are harmonized; each slide does not contain more than four colors; the color is constant in all slides; the amount of slide content is appropriate (maximum six word in each line and six lines per slide); bulleting is appropriate and does not make perceptional mistakes; title and content of slides are correlated; the ordering of slides is logical and the words and sentences are spelled correctly; grammar rules are observed.' The scale of giving feedback was "excellent, good, satisfying, needs to be improved and incomplete".

Evaluation
The evaluation was performed using the Kirkpatrick model at three levels: "Reaction, learning, and performance" (Figure 2) described in details as follows: After obtaining permission from the evaluation unit, the questionnaire was used in the study. The workshop evaluation questionnaire consisted of 11 items scored based on a five-point Likert scale (excellent, good, moderate, weak, and very weak) from 1 to 5. The questionnaire was completed at the end of the workshop personally by each subject.

Learning Level
The one-minute note technique was used to assess learning and stimulate the reflection of participants in the program. For this purpose, at the end of each workshop, two questions as "what were you interested in this workshop, and what you learned at this workshop that you use it as a professor in your teaching performance?" were given to the participants. In this technique, which is an active interactive lecturing technique, three main questions were asked: "What was the most important point you learned in today's session? What did you learn at this workshop that you use it as a professor in your teaching performance? What were the unclear points in the lecture?" There is no limitation for the number of questions that can be asked using this technique.

Performance Level
At this level, the outcomes of using interactive lecturing techniques were evaluated. For this purpose, six months after the workshop, the viewpoints of the faculty members participating in the program were asked through email to determine the extent to which interactive lecturing techniques were used in their classes. For this purpose, in a designed form, the participants were asked to note in the cell in front of each technique how much each technique was used in their classes within the last six months (number of times). In this regard, out of 140 emails, 64 completed forms were returned and the response rate was 45.71%. Email reminder was also sent to enhance the response rate.
Quantitative data were analyzed with SPSS version 22 (IBM Corporation, Armonk, NY) and one minute note sentences were also analyzed with manual content analysis. Regarding the nature of the data, the frequency (percent) and mean (standard deviation) were used for reporting.
Participation in the current study was completely optional; in addition, participants were free to complete questionnaire. The confidentiality of the names, the publication of the results in general and anonymous collection of information were among the ethical considerations in the study. Oral consent was obtained from all participants. This study was approved by HSR Committee of Education Development Center.

Response Level
Out of 140 participants, 120 completed the evaluation forms. Based on the evaluation results, most of the selected items were "excellent" and "good" (Figure 3).

Learning Level
Totally, 130 (92.5%) respondents answered the questions. Based on data analysis results by one-minute note technique, the workshop provided a new opportunity for faculty members to practice lecturing, reflect on the process of lecturing, more closely evaluate themselves, receive feedback, practice interaction in speech, and use effective lecturing principles. Participants believed that after participation in this workshop they more often tried using lecturing techniques, spend more time preparing their lectures, recording their speeches from now on, and watching the videos to resolve their weaknesses. Also, they may ask their peers to give feedback and change the educational contents. Table 1 shows an example of participants' statements.

Level of Performance
A total of 64 (45.71%) subjects responded to this question, and most of them selected Q& A, active evaluation, and use of scenario techniques to determine the mostly employed interactive techniques six months after participation in the workshop (Table 2).

Discussion
Due to the importance of using interactive lecturing in large classes and the importance of the empowerment of faculty members in this regard, the current study provided the experience of Tehran University of Medical Sciences in the employment of peer observation and feedback to empower faculty members in effective and interactive lecturing method. The results showed that the employment of peer observation and feedback to performance in empowerment programs along with the use of attractive and applicable topics such as oratory, vocal cords health, and problematic students could have a positive impact on the participants' viewpoints towards the workshop. The finding was consistent with the results of the study by Nasmith and Steinert (1). They believed that the way to hold faculty empowerment programs, in addition to influencing the participants' perspective and expanding the application of interactive and effective lecturing method, can affect the attitude and professional identity of the faculty members (1).
Based on the results of the current study, the content and manner of implementing the workshop was interesting for the participants and stimulated their reflection to use the contents of the program in their teaching practices. Analysis of the results by the one-minute note technique showed that the program could interest the participants to observe teaching and receive feedback from their peers. Today, peer observation and feedback is considered as one of the effective methods to empower medical education faculty members, which is expanding (10,11). Also, creation of this viewpoint in the faculty members that should assess themselves before making their speeches, in order to make their speeches interactive, and receive feedback of the others are among the most important outcomes of the implemented program, which reflects the stimulation of reflection on the teaching process and a sign of rethinking the teaching style.
In the study by McLeod et al. on the evaluation of the effect of peer observation program on the recorded speeches of professors, the program was highly accepted and led to the promotion of learning (7). Pattison et al. at McMaster University evaluated peer observation and feedback to enhance teaching abilities. Participants in their study believed that the provided feedback led to a rethinking of their professional performance (8), which was consistent with the results of the current study.
It is suggested that further studies should focus on ways to expand the employment of peer observation to enhance the quality of teaching. Also, using standard tools, further studies are suggested to evaluate rethinking as well as its depth in relation to the quality of teaching in faculty members.
Based on the results, among the 10 techniques presented in the workshop, the three techniques of Q& A, scenario, and active evaluation were used more than other techniques. To the authors' best knowledge, one of the limitations of the current study was that no information was found regarding the basic level of the employment of such techniques in the classroom. Perhaps the reason for more employment of such techniques is the familiarity of the faculty members with these methods, since Q& A and evaluation techniques are commonly used in lecture sessions. On the other hand, preconditions should be provided in order to employ some techniques. Therefore, none of the faculty members used flash cards. It seems that more sup-  Problematic students were interesting.
Vocal cords health was a new topic.
I had never seen videos of my lectures.
We have lots of problems …It is what I understand.
Techniques and their applications were new topics.
Giving and receiving feedback is difficult, but can be done.
The workshop was very practical.
What did you learn at this workshop that you will use it as a professor?
I try to record more videos from my lectures; there would be many points in them that I have never seen them before.
I will use interactive techniques such as scenario and debate.
I will look for receiving feedback, both in class and my slide set ...
I will activate lecture sessions and finish up monopole classes.
I will revise the contents and offer less content.
My slideshows should be made from the beginning.
The lecturing manner would be different in my classes.
port is required to provide physical space and facilities in order to expand the employment of interactive lecturing techniques. Nasmith and Steinert obtained satisfactory results in a similar program in which interactive lecturing barriers and strategies for interactive lecturing were introduced, and the opportunity to practice such strategies was provided during the workshop; according to their results, most of the participants, even after the program, used in-teractive techniques (1). It is suggested that other studies should also evaluate the impact of such techniques on the improvement of learning in students, and explore if the students are more satisfied with the professors that use interactive techniques? Does the use of such techniques precede the professor's degree of excellence and content expertise? Is it better to use a combination of techniques or any of the techniques separately? And which of such ap-  proaches can bring lifelong and deeper learning?

Conclusion
Based on the results of the current study, it seems that providing opportunities of practice, as well as observing performance and providing feedback can be effective on improving the quality of empowerment programs. Also, employment of empowerment programs using peer observation and feedback can have a positive effect on the satisfaction and learning of participants. It is also recommended that such tips be considered in other empowerment programs.
One of the main limitations of the current study was that the study only described an experience and presented the evaluation results. Therefore, it is suggested comparative methods as well as empirical and interventional researches be employed in further studies in order to examine the actual effects of empowerment programs based on the peer observation and feedback on professional identity, knowledge, and attitudes of the faculty members. The second limitation of the current study was the selfreporting of the participants about the number of techniques used. In this context, more accurate evaluations and measurements are needed to determine the impact of such and other similar programs. Also, the study was conducted only at one university, which restricts the generalization of its results.

Supplementary Material
Supplementary material(s) is available here [To read supplementary materials, please refer to the journal website and open PDF/HTML].