{'fraction_non_alphanumeric': 0.02396449704142012, 'fraction_numerical': 0.008994082840236686, 'mean_word_length': 5.128921124206709, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exploring Racism and Health: An Intensive Interactive Session for Medical Students

Introduction Growing recognition of the deleterious effects of racism on health has led to calls for increased education on racism for health care professionals. As part of a larger curriculum on health equity and social justice, we developed a new educational session on racism for first-year medical students consisting of a lecture followed by a case-based small-group discussion. Methods Over the academic years of 2016–2017, 2017–2018, and 2018–2019, a total of 536 first-year medical students participated in this mandatory session. The course materials were developed as a collaboration between faculty and students. The lecture was delivered in a large-group format; the small-group case-based discussion consisted of 10–12 students with one upper-level student facilitator. Results The majority of respondents for the course evaluation felt that the course had met its stated objectives, and many commented that they had an increased awareness of the role of racism in shaping health. Students felt that the small-group activity was especially powerful for learning about racism. Discussion Active student involvement in curriculum development and small-group facilitation was critical for successful buy-in from students. Additional content on bias, stereotyping, and health care disparities will be the focus of faculty development programs and will also be integrated into the clerkships to build on these important topics as students are immersed in clinical care.


Introduction
Recent events in the United States have brought to light the continued struggles the nation faces with respect to racial bias and racism. It is increasingly clear that experienced racism has a profoundly deleterious effect on the health of minority populations-at both individual and institutional levels. Minority populations are exposed to greater amounts of stress and adverse social determinants that can lead to higher risk for poor mental and physical health. In the US health system, minority patients are less likely to receive high-quality health care services across multiple clinical settings, driven in large part by implicit bias on the part of physicians.
Throughout medical training, students and trainees are exposed to multiple methods of instruction that enhance bias and misconceptions about race, genetics, culture, and disparities.
In this context, teaching medical professionals about structural racism and how to recognize and address bias in clinical encounters has become increasingly imperative.
Physicians, public health advocates, and medical educators have long recognized the impact that bias has on the well-being and care of patients. In acknowledging this, the Liaison Committee on Medical Education's Accreditation Standard 7.6: Cultural Competence and Health Care Disparities requires that "the faculty of a medical school ensure that the medical curriculum provides opportunities for medical students to learn to recognize and appropriately address gender and cultural biases in themselves, in others, and in the health care delivery process." Likewise, medical students are increasingly requesting a focus on health equity and racism in medical education, including at our institution. In 2014-2015, Rutgers New Jersey Medical School (NJMS) students, in response to multiple killings of unarmed black men by the police, staged a die-in as part of the White Coats 4 Black Lives movement and advocated for increased teaching on issues of racism and its effect on health in the traditional NJMS curriculum.
In response, as a part of major curriculum revision, in academic year 2016-2017 (AY17), we created a new required course, Health Equity and Social Justice (HESJ), designed to be longitudinal over the preclerkship years, with the goal of making health equity content in the curriculum more accessible and explicit. The course included sessions on power and privilege in the physician-patient relationship, the role of unconscious bias in health care, trauma-informed care, spirituality, the social determinants of health, working with underserved populations, and exploring the patient's perspective, among others.
We offered the 3-hour, mandatory HESJ session on racism and health to all first-year medical students within the first month of the curriculum over AY17, 2017-2018 (AY18), and 2018-2019 (AY19). The session consisted of a didactic lecture and a small-group case-based discussion. Prior to the session, all students had attended an interactive lecture on unconscious bias, completed an Implicit Association Test (IAT), and submitted a personal reflection essay on the results of the IAT as part of HESJ.
Prior work with educational modules on racism for medical students, psychiatry residents, third-year clerkship students, and physicians has shown an increase in awareness of the importance of racism and bias, as well as an enhancement of a commitment to health equity. These interventions used one of multiple different modalities, including didactic lectures, small-group case-based discussions, online tutorials, and IATs. We found four such publications in MedEdPORTAL, one of which was designed for preclinical medical students but none of which included both didactic lecture and small-group casebased discussions.
Unlike other MedEdPORTAL publications, we incorporated both large-group didactic and case-based small-group activities to increase awareness of how racial historical insults shape society and consciousness, as well as to guide students in absorbing and processing the new information through debate and reflection. We engaged students in the curriculum design, a technique that has been successful in other efforts addressing cultural competency training. In addition, because we based the cases on student experiences, the discussions provided students opportunities to brainstorm about skills that they could use to navigate bias when they experience or witness it in the clinical setting.
We chose to expose students to these topics very early in their clinical training (the first month of the first year) to set the stage for subsequent clinical experiences and for future health equity curricular elements.
The topics included the power and privilege dynamic in the clinical encounter, cultural humility, social determinants of health, working with underserved populations, and adverse childhood experiences/trauma-informed care-all areas where race and racism play important roles influencing health and access to high-quality health care.

Recommended Background Resources
To prepare for the session, we recommended that students read one article and view one video. Katherine Brooks' perspective piece "A Silent Curriculum" reflects on her medical school experience and provides first-year students with an introduction to the role bias plays in health care disparities. Dorothy Roberts, in her TEDMED talk "The Problem With Race-Based Medicine," explores the history of racebased medicine, its continued influence on physicians' decisions and health disparities, and the importance of recognizing race as a social concept.

Didactic Lecture
The lecture, detailed in the lecture slides (Appendix A), required the use of a large lecture hall with a computer and projector with PowerPoint. We allotted 50 minutes for the lecture and provided a detailed materials checklist and time line (Appendix B). We chose to include a didactic component to introduce students to concepts and historical perspectives, including the history of the establishment of our institution and its impact on race relations in Newark, NJ, that have shaped the face of modern racial bias. We referred to the revised Tool for Assessing Cultural Competence Training in the early planning stages of creating learning objectives.
The lecture explored the changing definitions of race over time, differentiated the levels of racism, provided examples of how health can be affected by each level, and explored ways to address racial bias in health care. We used Camara P. Jones' work on race and the levels of racism as a key resource for development of the lecture materials. We drew from prior work in MedEdPORTAL to describe the historical context influencing racial/ethnic health care disparities. We provided more in-depth examples of the interaction between historical and present-day racism and health for American Indian/Alaska Native and African American/black populations. The addition of a 2018 study discussing the spillover effects of police killings on black Americans was included in AY19.
We also incorporated into the lecture approaches to addressing bias in clinical encounters. We had found in the first 2 years that frameworks and suggestions were needed to enhance the discussion and to give students practical and portable tools that could help them address bias and microaggressions when encountered in clinical work. In AY19, we incorporated two portable mnemonic frameworks to address both personal unconscious bias and external microaggressions.
We developed the CHARGE2 framework, which described efforts health care workers could make to mitigate the effects of their own unconscious bias. The framework is as follows: • C-Change your context: Is there another perspective that is possible? • H-Be Honest: With yourself, acknowledge and be aware. The framework summarized key points from a previous HESJ lecture on unconscious bias by author James Hill, a trained clinical neuropsychologist and the associate dean of student affairs at our institution. The framework drew on several resources in developing steps to mitigate implicit bias. CHARGE2 was first piloted in this session in AY19 and was not modified over time.
We adapted the INTERRUPT framework with permission to provide students with a tool kit of responses with specific language to address bias and microaggressions. We hypothesized that this framework would be especially useful for students who might find it difficult to speak up due to power dynamics within teams in academic medicine. It was first piloted in this form in the AY19 session. However, we had piloted excerpts from the Interrupting Microaggressions framework during an orientation for clerkship students 3 months earlier. Preliminary feedback from the orientation session showed that the students appreciated having tangible tools for addressing bias on their clinical rotations.
The INTERRUPT framework is as follows: • I-Inquire: Leverage curiosity. "I'm curious, what makes you think/say that?" • N-Nonthreatening: Convey the message with respect. Separate the person from the action or behavior. "Some may consider that statement to be offensive." Communicate preferences rather than demands. "It would be helpful to me if. . . ." • T-Take responsibility: If you need to reconsider a statement/action, acknowledge and apologize.
Address microaggressions, and revisit them if they were initially unaddressed. • E-Empower: Ask questions that will make a difference. "What could you/we do differently?" • R-Reframe: "Have you ever thought about it like this?" • R-Redirect: helpful when individuals are put on the spot to speak for their identity group. "Let's shift the conversation. . . ." • U-Use impact questions: "What would happen if you considered the impact on . . . ?" • P-Paraphrase: making what is invisible (unconscious bias) visible. "It sounds like you think. . . ." • T-Teach by using "I" phrases: Speak from your own experience. "I felt x when y happened, and it impacted me because. . . ." Small-Group Case Discussions A group of upper-level students collected the case studies for the small-group session. The students who contributed cases were recruited by a senior student (author Megana Dwarakanath) who had expressed interest in health equity topics to the course director and collated the cases for the course director to review. The cases were based on clinical experiences that led a student to struggle with concepts of race or racism. Because the case studies were gleaned from actual student experiences, they were designed to challenge students to look more closely at their own assumptions about the proximity of race and racism impacting their training and patient care. We also hoped that speaking openly about examples of racism would empower students to share their own experiences of noting racial bias in patient-physician encounters.
We created a working group of four students and the course director to develop prompting questions and learning points for each case. We recommend using the case development worksheet (Appendix C) as a guideline to effectively develop cases. We developed this worksheet based on our own experience in collecting cases, analyzing themes, and deciding on relevant learning points. Five cases with different themes were ultimately chosen. The facilitator guide and the worksheet were the result of brainstorming sessions and discussions among the students based on their experiences and insights, rather than being based on a literature review. The lecture notes detailed in the facilitator guide (Appendix D) were added in AY19 and were largely based on student peer facilitator feedback from the prior 2 years. The course director further refined the prompting questions and facilitator notes informed by narrative medicine and perspectives from the medical literature.
We chose to have student peers, rather than faculty, facilitate the discussion to create a safe space for students to debate openly without fear of being assessed and without worry of trying to come to a consensus that would be acceptable to an authority figure. We also felt that student facilitators could more proximally give advice to students about how to navigate the medical hierarchy. These ideas emerged as a result of reflection and feedback from the student volunteers who assisted with curricular development for the HESJ course and the session on racism in particular.
The student peer facilitators were recruited through personal invitation from the core student working group as well as through class-wide emails requesting volunteers. The facilitators were second-, third-, and fourth-year students. Prior to the session, the peers attended a 60-minute evening training where the course director reviewed the facilitator guide and discussed the cases and prompting questions with the group. This training was attended by approximately 50% of the peer facilitators each of the 3 years. We did not collect data on the demographics of the peer facilitators.
Each small group consisted of 10-12 students paired with one peer facilitator; the small groups met after the lecture, with a 10-minute break between the lecture and the small groups. The materials needed for the small-group discussion included rooms with a table and chairs, one printout of the facilitator guide, and printouts of the student pages (Appendix E), one for each student (10-12 per group). The facilitators handed out student pages at the beginning of the session. First, the groups reviewed the ground rules; then, they discussed each of the cases using the prompting questions. We allotted 120 minutes for discussion of the cases and prompting questions; most groups finished the discussion of all five cases within 90-100 minutes. We indicated no prespecified break; however, students were allowed to leave the room for short breaks as needed.
The facilitator guide also included notes for each case detailing specific concepts and language that could be used to address biases and microaggressions, as well as a description of practical tools and takeaway messages that could assist students in navigating difficult scenarios they might encounter. Highlighted skills included reflection using the CHARGE2 and INTERRUPT frameworks, as well as advice on where to seek support and guidance. Appendix B provides a materials checklist and time line for the small groups.

Assessment
Assessment of the session initially consisted of multiple-choice questions testing content from the lecture incorporated into unit exams in the systems-based curriculum in AY17. However, students in both the course evaluation and focus groups felt that this approach was stress provoking and took away from broader, deeper themes that could not be assessed with multiple-choice questions. Based on student feedback, starting in AY18, we administered the questions instead as an online, open-content quiz (Appendix F) testing content of the lecture and the small group. The course director developed the quiz questions-four multiple-choice questions based on the lecture content and one short-answer question based on the small-group content ("What top two messages did you take from the case discussions during the small-group session on racial bias medicine and why?"). Over time, we further developed the multiplechoice questions to more closely match the course objectives.
The course director graded the quizzes and incorporated the score into the overall grade for the HESJ course.
The short-answer responses were graded based on demonstrating the following: (1) understanding of the content, (2) thoughtfulness and self-reflection in the responses, and (3) evidence of having engaged with the topics of discussion and resources. Students were provided with these criteria in advance in the course syllabus and during the course introduction.

Evaluation
We provided an HESJ course online anonymous evaluation form to all students at the end of the first 2 academic years through an internal educational management system. Students were asked to reflect back on all of the sessions in the course and report on how well each session met its learning objectives. They were also asked to provide any comments about how the sessions and overall course could be improved.
In those first 2 years, the evaluations were submitted approximately 9-10 months after the session and also asked students to reflect on multiple other educational sessions in the HESJ course, limiting the number of questions that could be asked about each individual educational component. We found that response rates were low (less than 50%).
In AY19, we expanded the evaluation to include more detailed questions as well as free-text responses about strengths and improvements. The students submitted the evaluations online using Qualtrics as a survey platform. We asked the students to submit responses within 1 week after the session. The AY19 evaluation form is included in Appendix G.
Annual end-of-course focus groups were conducted for AY17 and AY18 to ensure ongoing student involvement in any course improvements and changes. We recruited eight first-year participants in AY17 and seven in AY18 via a class-wide email invitation and then by interested students who recruited additional participants. These self-selected students demonstrated an interest in health equity topics and provided predominantly positive feedback about the session. They reported that the presence of peer facilitators enhanced their comfort level during the discussion. In AY17, they also suggested a change to the online open-content quiz format and overwhelmingly supported incorporating questions into the unit examinations, particularly since they speculated that a short-answer component could stimulate selfreflection and a deeper understanding of the content.

Data Analysis
We were granted institutional review board approval for the analysis of evaluation and quiz data. We extracted the anonymous evaluation data for each year from the online surveys. For the strengths and improvements session on the AY19 evaluation, we categorized, coded, and quantified the themes. For the short-answer quiz question from AY18 and AY19, we downloaded the responses from our online educational platform, permanently removed student identifiers (names), and then categorized, coded, and quantified the themes expressed for each response. On average, each individual student response contained 2.9 and 3.1 themes in AY18 and AY19, respectively. Two of the authors (Michelle DallaPiazza and Mercedes Padilla-Register) independently analyzed the text evaluation and quiz comments and responses and then highlighted and discussed any differences to come to a consensus. Disagreement in the coding of the responses occurred less than 5% of the time.

Student Demographics
In the 3 years that we offered this mandatory session on racism and health, a total of 536 students participated. The demographics of each class by gender, race, and ethnicity are summarized in Table 1. Analysis of Student Evaluations At the end of HESJ AY17 and AY18, 55 (30%) and 76 (41%) students, respectively, completed the course evaluation form. Overall, for AY17 and AY18 combined, 95 (72%) felt that the learning objectives for the session had been met to a considerable or a very high degree. Thirty-two (25%) felt that the learning objectives had been met to a small or moderate degree, and four (3%) felt that the learning objectives had hardly been met at all.
In AY19, 165 (92%) students completed a more detailed session-specific evaluation form. The responses focusing on the learning objectives are summarized in Figure 1. The students largely felt that their knowledge and skills improved with respect to the learning objectives; however, the learning objective that needed additional exploration compared to the others was defining race and racism. The students reported that each of the components of the activity also contributed to their learning to high degrees ( Figure 2), with the group discussion and small-group facilitator effectiveness rated most favorably. Lastly, 63% agreed to a great or considerable degree and 26% to a moderate degree that they felt more comfortable using CHARGE2 and INTERRUPT, and 71% agreed to a great or considerable degree and 21% to a moderate degree that additional training on this topic would be beneficial to their learning to become doctors ( Figure 3).   Regarding the strengths and improvements section of the AY19 evaluation (summarized in Table 2), positive comments about the session focused on several themes, mainly on increased awareness and an appreciation for interactive and collaborative learning in the small-group discussion session. In the first 2 years, critical feedback mainly centered on the fact that HESJ sessions in general distracted the students from other obligations and pressures. With the more detailed AY19 evaluation, however, we obtained richer feedback. These comments dealt mostly with making the lecture more high yield and interactive and with spending more time exploring the medical hierarchy, with actionable points on how to address bias from the perspective of a medical student with seemingly little power in the team dynamic. More time in small groups/less in lecture 18 "The lecture was long and cut into the time meant for small-group discussion, which I found to be the most valuable part." More time in lecture to explore concepts in depth/make lecture more interactive 15 "I think that Tuskegee trial should be talked about in much more depth . . . especially as students may wish to pursue clinical research." Small group More cases/more diverse or realistic cases/less obvious "right" answer 15 "The small-group cases could have been a little more provocative-I felt like some of them were too blatantly wrong . . . and it would be better for students to be exposed to the less obvious instances." More actionable points/more on navigating medical hierarchy 13 "Without good examples of how these problems are being addressed and how the attitudes are changing in healthcare, it will feel like a hopeless task." N indicates the number of responses fitting a theme.

Analysis of Quiz Responses
For AY18 and AY19, all students ( n = 358) completed the quiz. The average grade for the quiz was 95% in both years. We summarize the results of the analysis of the short-answer question in Table 3. For both years, the frequencies of themes were similar. The most frequent learning topics focused on increased awareness about internal bias, the importance of addressing bias, and the impact that bias has on patient care and health systems. In addition, students noted that the culture of medicine and the medical hierarchy in medicine can create challenges to addressing bias in ways they had not previously considered. 46 (26) 40 (22) Conversations uncomfortable but essential/helpful to discuss scenarios prior to immersion in clinical care/important to be prepared for microaggressions.
39 (22) 34 (19) Patient bias against providers can affect the quality of care/challenging to address without alienating patient. 29 (16) 39 (22) Racism can cause power shifts within teams of health care workers. 13 (7) 6 (3) Powerful that scenarios were based on real experiences of students. 9 (5) 9 (5) "Othering" and "us vs. them" attitudes can be detrimental to patient care. 9 (5) 4 (2) Patient-related Important to engage with patients to understand their experience with discrimination in health care/enhance trust/empower patients. 31 (17) 18 (10) Health care is a right, not a privilege/race is a social construction that unfairly disadvantages patients. 10 (6)  The CHARGE2 and INTERRUPT frameworks were explicitly highlighted in the lecture and small groups in AY19 only; the quiz responses for AY19 frequently emphasized the usefulness of the frameworks for addressing both internal and external bias (21% and 38% of responses, respectively). Overall, in addition to highlighting key learning points, the responses often demonstrated the students' capacity for introspection and empathy for patients and peers.

Discussion
Racism, because of its impact on health and health care, should be explored by all medical students as part of their educational journey to become a physician. Our effort, detailed here, was successful at meeting its goals of exploring the levels of racism, reviewing the implications of historical racism, facilitating reflection and dialogue, and introducing skills that can be developed to work against bias.
Unlike other approaches, we chose to include a didactic lecture prior to engaging in a small-group discussion in order to demonstrate that the legacy of racism influences the unconscious bias in everyone. The knowledge that racial bias affects many aspects of health and permeates the physician-patient relationship proved to be eye-opening for many students and was important for framing the small-group discussions that followed. In future years, based on student feedback, we will add more emphasis on defining key terms, such as race and racism, and on exploring the team dynamics of the medical hierarchy in more depth.
The lecture, while important, is one reason why we dedicated 3 hours to this learning session. Since the group discussion provided the most robust learning experience, for others hoping to implement a similar session in less time, we recommend either assigning the lecture slides in advance or selecting key slides to reduce the lecture time to 15-20 minutes. As an example, the lecture focuses in depth on historical instances and on describing and providing evidence for institutional racism; these concepts could be significantly abbreviated to concentrate more on differentiating the levels of racism and on the tools that can be used to combat racism in clinical practice. The case discussions could also be abbreviated by limiting them to two or three high-yield cases based on institutional needs.
In the small groups, the first-year students highlighted the opportunity to engage in a discussion incorporating multiple perspectives and viewpoints of their peers as a major strength of our approach. They were also able to identify several barriers and strategies to overcoming these barriers through use of the CHARGE2 and INTERRUPT frameworks. The incorporation of these elements in the most recent iteration of the session (AY19) helped the students to apply concrete steps for self-reflection and action to address bias. In both the evaluation and, especially, the quiz responses, these frameworks emerged as important tools that helped to frame the discussion and generate actionable points.
Upper-level students proved to be an invaluable resource in the development and implementation of the curriculum and, in particular, in the inclusion of potentially highly charged topics such as racism and bias in medicine. By leveraging the expertise of student advocates to design small-group case-based discussions based on real experiences, we aimed to create a safe learning environment that provided a basis for personal growth through reflection. In addition, because the upper-level students' experiences were more proximal and relatable, students valued their reflections and sharing of lessons learned. Student participants, the student peers who led the conversations, and the focus groups of first-year students affirmed that the facilitators were successful at engaging students and, because faculty were not present, created a safe space for students to share a broad range of opinions and reflections.
An important limitation to this approach is that we excluded from the case discussions a significant contribution to the learning environment-faculty. Faculty need to be part of the dialogue around racism and bias in medicine ; they are the mentors and role models from whom students learn clinical behaviors. Faculty require training and exposure to comfortably and meaningfully engage students on these topics. To address this need, we have met with each faculty course director in the preclerkship curriculum to discuss strategies to avoid inclusion of potentially biased information in educational content and to enrich activities with health equity topics. We have also developed a core group of faculty facilitators from multiple specialties in a learning community (LC) for the preclerkship curriculum. We train LC facilitators in delivering the HESJ small-group content and encourage them to attend the lectures even if they are not involved in facilitation of the corresponding small group. We are also currently implementing faculty enrichment programs on unconscious bias and racism beyond the LC.
An additional limitation to our approach is that both the lecture and the small-group discussion focused mainly on American Indian/Alaska Native and African American/black health disparities, while only touching briefly on Asian American or Latino(a) American experiences with racism and discrimination. Especially in light of the overall demographics of our student body, this limitation may be significant, since we miss out on an opportunity to glean insight from a wider section of the class. Future work could include more information on the effect of experienced racial and ethnic discrimination on health for other minority groups. A recent meta-analysis showed that while racism was associated with poorer health for all groups, the association between racism and negative mental and physical health was significantly stronger for Asian American (for mental health only) and Latino(a) American (for both) when compared to African American/black participants.
We felt it was important to incorporate assessment into the HESJ course at large, so we included quiz questions in the course design. When converting to an online, open-content format, we found that the grade averages were very high-95%. However, we were able to glean important information about learning topics from the small-group sessions and introduce another layer of content review to cement learning of the concepts. Future work could focus on assigning HESJ content more value in the overall medical education curriculum, which could be facilitated by advocacy to include social medicine topics in the USMLE examinations.
As highlighted by student feedback, it is critical to ensure that we revisit HESJ content later in training, particularly at the time when students are immersed in clinical care and exposed to the reality of health disparities. Because the data we present here are based on student perceptions, we cannot be sure how this training will later translate into behaviors or how durable the knowledge and skills will be. We will aim future work at developing longitudinal HESJ content spread throughout the clerkship year. In AY18, we piloted interactive sessions at the beginning, middle, and end of the clerkship year on bias and stereotyping in clinical care and evidence-based approaches to address health care disparities. New objectives and questions focused on bias have been added to each clerkship evaluation, and currently underway is curriculum development driven by the clerkship directors in each specialty that touches on at least one HESJ topic, such as social determinants of health, health care disparities, and trauma-informed care.
We specifically designed the materials in this session for first-year medical students. However, because of the generalizability of the content, the session can be adapted to other groups and settings, including other health professional students, residents, and faculty, as an introduction to racism and health. We anticipate that the skills concerning personal reflection and nonthreatening approaches to addressing bias will prove useful in the clinical care of patients and in supporting colleagues. As health care shifts to place greater emphasis on population health and community engagement, curricula such as this, which stimulate greater awareness and advocacy among future physicians, are a critical first step in promoting health and wellness for all communities, including those most vulnerable to bias and discrimination.