{'fraction_non_alphanumeric': 0.024663606142156087, 'fraction_numerical': 0.005034035143264208, 'mean_word_length': 5.328591464636346, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Medical Student Consulting: Providing Students Leadership and Business Opportunities While Positively Impacting the Community

Introduction Medical schools are increasingly attempting to prepare future physicians for diverse new leadership roles in the health care system. Many schools have implemented didactic leadership curricula, with varying levels of structure and success. Project-based learning via completion of real-world projects using a team-based approach remains an underutilized approach to developing student leadership skills. Methods We designed and implemented the Medical Educational Consulting Group (Med ECG)—a student-run consulting program that provides medical students with opportunities to develop leadership skills by completing consulting projects with local clients. We provide an overview of the Med ECG model, including a combination of didactic training sessions and project-based learning via both simulation and real-world projects. Surveys were used to evaluate the value of Med ECG to clients, the community, and students. Results Fourteen medical students (eight first-years, two second-years, three third-years, and one fourth-year, including five dual-degree candidates) completed the Med ECG program. Client feedback pointed to the value of Med ECG's projects and their impact on the community through partner organizations. Finally, linear regression analysis showed a strong positive correlation (R2 = .61) between the amount of effort devoted to various leadership attributes and the perceived improvement while working with Med ECG. Discussion Med ECG's experiences show that a medical student-led project-based learning program is a novel way to develop leadership skills for future physician leaders. Efforts to replicate these types of programs will help additional students develop their leadership and business skills, making a positive impact outside of the classroom.


Introduction
Modern health care increasingly requires a team approach. Leadership of teams is becoming an essential function for physicians, but many physicians have little or no leadership training. Physician leaders can have a positive impact at a variety of levels in the health care system, but in 2014, only 5% of hospital leaders were physicians. 1 However, 10 of the 18 "Honor Roll" hospitals according to US News & World Report rankings from 2013 were physician led. 1 Moreover, it has been found that US hospitals with physician chief executive officers are associated with superior quality according to their hospital ranking. 2 As health care in the United States continues to evolve, it will become increasingly important to have physicians at the forefront of hospital and health system leadership to encourage changes that not only reduce costs but also improve patient care.
Many resident, faculty, and medical student leadership programs have been established to provide physicians with essential leadership skills. [3][4][5] Business and management skills are a key part of leadership training. A survey of 18 medical schools showed that two-thirds of medical students perceived a background in management and business to be an important aspect of their roles as future physicians and were interested or highly interested in learning more about business in medicine. 6 Many schools have implemented didactic leadership and business curricula, with varying levels of structure and success. [7][8][9] There have also been successful extracurricular attempts to develop physician leaders, supporting the idea that project-based leadership development can improve leadership training. [10][11][12][13] Whereas several previous MedEdPORTAL publications on leadership and business have focused on didactic curricula primarily aimed at residents, [14][15][16] we introduce a project-based learning program tailored for medical students.
To address the described gap in medical education, we founded the Medical Educational Consulting Group (Med ECG) in 2017 at the University of Michigan Medical School with two primary aims: (1) provide medical students with didactic training and projectbased learning experiences to enable them to become future leaders in health care and (2) improve the community through projects with local nonprofit health care organizations. Herein, we describe the establishment and growth of Med ECG within our own institution. We provide content and advise readers on how to practically implement a similar program at their home institutions. Last, we describe Med ECG's challenges, lessons learned, and generalizability.

Methods
Establishing Med ECG In 2017, Med ECG was founded by four students with 1-3 years of prior consulting and/or corporate health care experience (David S. Portney, Paige VonAchen, Taylor Standiford, and Matthew R. Carey). The overarching goal was to facilitate students to complete pro bono projects with local nonprofit health care organizations to improve community health and organization sustainability. To establish the organization, we filed as a Voluntary Student Organization at the University of Michigan.

Recruiting
To recruit medical students, we held a 1-hour information session, in which we discussed the mission of the organization and the types of projects we planned to facilitate and presented a highlevel overview of the training students would receive. After the information session, students applied to join by emailing the founders with answers to the following two questions: (1) Why are you interested in joining Med ECG? (2) Why do you think you would be a good fit for Med ECG? All students who applied had a 30-minute interview with the founders, in which we asked applicants about their goals and interests in terms of business and leadership skills they wished to develop through participation in the organization. The data from these interviews were then used as inputs in determining the assignment of students to particular projects. During the first year, all students who applied and expressed both interest and a willingness to commit during interviews were accepted to participate in the program.
Participating students had varying levels of experience, ranging from no formal business experience or education to prior professional business experience and advanced business degrees. Students across all levels of medical school were involved in the program, and each project team included preclinical and clinical students. In Med ECG's first year, there were 14 members, including eight first-year students, two second-year students, three third-year students, and one fourthyear student. Of the 14 total members, five of the students were pursuing secondary graduate degrees, including three MBA students and two PhD students. In relation to prior business or health care experiences, five had a significant amount of experience, six had an intermediate amount (e.g., working on an undergraduate health policy research project), and three had very little or no prior experience.

Training
With a wide range of incoming skill levels, the leadership and technical skills training was designed to benefit students with varying levels of prior business experience. The training began with two 1-hour didactic training sessions and a third session with students broken into smaller groups to develop baseline skills: (1) introduction to consulting and problem solving, (2) data analysis and sharing findings, and (3) teamwork and presentation skills. The first two sessions took place on a Thursday evening, so as not to interfere with lectures or clinical work that students participated in during the day. The last sessions were individually scheduled by the groups according to each member's schedules. Below are descriptions of each of the three sessions and a description of how to implement them using the materials provided. Note that all patient names and identifications were made up for the purpose of the exercise.
Introduction to consulting and problem solving: During this training session, students were introduced to a sample consulting project, guided through the problem-solving framework, and given a preconstructed database based on the sample consulting project.
r Materials to facilitate: • Appendix A: training session 1 video file.
• Appendix B: introduction to consulting and problemsolving session slides.
• A computer with Appendices B and C available.
• A projector and screen on which to project the slides.
r Instructions to facilitate: To implement this session, the facilitator should become familiar with the content within the introduction to consulting and problem-solving session slides (Appendix B). Within the appendix, each slide contains talking points in the Notes section of the slide. Next, the facilitator should open the preconstructed database (Appendix C). The facilitator should be familiar with basic Excel skills, including pivot tables, IF function, SUM function, Insert Table function, formulas, and filters. The facilitator should open Appendix A and watch the video of one of our founders facilitating the session with our students. By watching the video and becoming familiar with the talking points on the slides, the facilitator should feel comfortable implementing the session at his or her institution. At the end of the session, the facilitator should instruct the students to use the Excel skills taught in the session to attempt to find patterns in the data that may be beneficial to the mock client in the sample consulting project. Thus, students simulate solving the sample consulting project with the preconstructed database.
Data analysis and sharing findings: During this training session, students were given an Excel tutorial using the preconstructed database. Next, the facilitators provided didactic training on slide writing (Appendix F). Last, the students were provided with a set of sample slides (Appendix H) and were placed on teams for their final presentations.
r Materials to facilitate: • Appendix C: preconstructed database.
• Appendix D: training session 2 video file-Excel walkthrough.
• Appendix E: answers to the preconstructed database.
• Appendix F: training session 2 video file-PowerPoint walk-through. • Appendix G: data analysis and sharing findings session slides.
• Appendix H: set of sample slides. • A computer with Appendices C and G available.
• A projector and screen on which to project the slides.
r Instructions to facilitate: To implement this session, the facilitator should become familiar with the content within the data analysis and sharing findings session slides (Appendix G). Within the appendix, each slide contains talking points in the Notes section of the slide. Next, the facilitator should open the preconstructed database (Appendix C). Again, the facilitator should be familiar with basic Excel skills, including pivot tables, IF function, SUM function, Insert Table function, formulas, and filters. The facilitator should open Appendix D and watch the video of one of our founders facilitating the Excel training portion of the session with our students. Next, the facilitator should open Appendix F, which is a video of one of our founders facilitating the slide writing portion of the session with our students. By watching the videos, becoming familiar with the talking points on the slides, and understanding how to obtain the answers to the preconstructed database (by going through the steps outlined in Appendix D), the facilitator should feel comfortable implementing the session at his or her institution. At the end of the session, facilitators should assign students to teams of two to four and email all students the set of sample slides (Appendix H). The sample slides are for students to use as a resource when developing their final presentations and for future use with clients (and beyond).
Teamwork and presentation skills: Prior to this session, students worked among themselves (on their own time) with the teams they had been placed on during session 2 to create final presentations based on the sample consulting project. They were instructed to develop a presentation using the answers to the Excel tutorial provided in session 2 and deliver final recommendations based on the sample consulting project. During the session, each team provided a final presentation to the entire group of students and the facilitator.
r Materials to facilitate: • A computer with the final presentations available.
• A projector and screen on which to project the slides.
r Instructions to facilitate: Prior to this session, the facilitator should instruct the teams to create presentations based on the sample consulting project using the answers to the Excel tutorial provided in session 2 and to deliver final recommendations based on the sample consulting project. Each team should email its final presentation to the facilitator prior to the session. During the session, the facilitator introduces each of the teams and observes each team's presentation. At the end of each presentation, facilitators should provide each team with feedback on its presentation. Feedback may include comments regarding the quality of slides, delivery of presentation, and so forth.

Client Development
We identified local prospective client partners through faculty connections, online outreach, and word of mouth. Prospective clients were defined as local nonprofit health care organizations. To develop client relationships, we emailed seven prospective organizations and set up a phone-based conversation, followed by an in-person meeting. The phone-based conversation was used to discuss the mutually beneficial goals of the program and brainstorm potential projects based on the clients' needs. During the in-person meeting (the scoping meeting), a specific project was selected based on a dynamic discussion between the founders and the clients, incorporating both student interests (as ascertained during the interview process) and client needs. Additionally, care was taken to discuss students' level of training and the 4-month project time frame and to define the specific deliverables that would be expected at the end of the project. These discussions ensured that there were clear expectations from both the clients and the students regarding the scope of work expected. The meetings took place in parallel with new member training to ensure that projects could begin shortly after training completion.
The specific client selection occurred on a case-by-case basis, much of which was organic. Of the seven prospective organizations contacted, two did not respond at all, two responded that they would be interested at a later date, and three became clients. In the future, the goal is to build on the relationships of current clients to streamline the process and standardize the experience for future students.
Project-Based Learning Through Community-Oriented Projects Following the foundational skills training, a majority of the leadership, professionalism, and technical skills were developed through project-based learning on local community projects. Project teams consisted of four medical students, including a project leader with a proven level of leadership experience and business acumen. Teams had both preclinical and clinical medical students, providing a unique learning environment that is often nonexistent in other areas of medical school.
The project leads were responsible for developing the project time line, project management, and ongoing client communication throughout the 4-month period. Most projects had one to two in-person client meetings throughout the 4month project period, during which the team would present project updates, project-specific analyses, and draft deliverables (as defined during the scoping meeting). Depending on the project, there was a varying range of phone-based client meetings in addition to individual interviews with relevant stakeholders. The project lead determined the frequency of client communication and meetings based on project needs. Team members contributed to the project through individual work streams, as defined by the project leads. The project leads were responsible for determining the optimal approach for assigning responsibilities to individual team members to maximize the quality of output for the clients while simultaneously providing team members with learning opportunities to expand their skill sets. For example, a project lead might have asked two students to work on a data analysis, one of whom had previous Excel experience and one of whom had none, for the new student to learn the skill while not jeopardizing the quality of work provided to the client. The projects concluded with valuable and implementable deliverables (as defined during the scoping meeting) in the form of a final in-person or virtual presentation to the client leadership.

Evaluation of Med ECG
We measured the success of Med ECG through three main outcomes. The first outcome was how well Med ECG fit into the established guidelines and competencies for leadership and business acumen training. The second outcome was the extent to which the work produced was valuable for client partners and the community. The third outcome was the ability of Med ECG projects to promote the development of leadership and business skills for its members. Next, we describe the development of instruments for each outcome and the subsequent analyses performed: r Outcome 1: Relevance to Existing Leadership Competencies-The first outcome was analyzed qualitatively using the framework outlined by Webb et al. 7 In this review, medical school leadership curricula were compared using the Medical Leadership Competency Framework (MLCF) 17 and Kirkpatrick's four-level training evaluation method. 18 We analyzed our own program's leadership competency across the same dimensions to determine where Med ECG fit with other medical school leadership curricula. The five components of the MLCF are demonstrating personal qualities, working with others, managing services, improving services, and setting direction. 17 The four levels of Kirkpatrick's model are Level 1: reaction, Level 2: learning, Level 3: behavior, and Level 4: results. To examine where Med ECG fit into Kirkpatrick's four-level training evaluation method, we examined the scope of feedback from Med ECG team member surveys, by which members self-rated their experiences and training across 10 leadership, professionalism, and business skill attributes.
r Outcome 2: Client Perceptions of Med ECG Impact-The second outcome was measured using a client survey focused on the Med ECG team's professionalism and communication skills, the project's scope, the value of the project's output, and the organization's feedback for Med ECG (Appendix I). Eight metrics were used to determine the value of the project's output: research findings, data analysis, prioritization and segmentation techniques, consolidation of sources, practicality of solutions, effectiveness of solutions, and presentation of recommendations. Clients were asked to rate the team's output and skill on a 5-point Likert scale, with 1 as the worst and 5 as the best. We used these descriptive statistics to analyze the impact of Med ECG on partner organizations. r Outcome 3: Student Perceptions of Learned Skills and Improvement-The third outcome was measured using a team member survey focused on specific skill development, value gained, and feedback for future projects (Appendix J). To measure skill development, 10 leadership and consulting skill areas were defined. For each skill, team members were first asked to rate how much their project allowed them to utilize or build on the skill. For this question, respondents were given a 5-point Likert scale (1 = very little or none, 5 = significant time or effort). Next, for each skill, team members were asked to rate the perceived improvement in their own skill level. This question used a 5-point Likert scale (1 = ability got worse, 2 = no change, 3 = slight improvement, 4 = modest improvement, 5 = significant improvement). We then performed a secondary analysis of team member development to determine the extent of improvement based on active experiences. We compared each team member's effort on a specific attribute during the project to perceived improvement on that specific attribute.
Both surveys were administered through Google Forms (Google Inc., Mountain View, California) and completed within 1 month after the completion of the projects and the project symposium. All statistical analysis was completed using Stata 15 (StataCorp LLC, College Station, Texas) with a p value of .05.

Materials
The materials to successfully administer this project-based leadership curriculum have been outlined above in the Training section. However, to summarize, facilitators should have the following materials: r A computer with access to the training session slides (Appendices B and G).
r Preconstructed database for data analysis training (Appendix C).
• Note: The facilitator should feel comfortable with the skills outlined in the Excel tutorial (as delivered in Appendix D). Specifically, the facilitator should be able to use the following: pivot tables, IF function, SUM function, Insert Table function, formulas, and filters.
r A projector and screen on which to project the slides. r Identified local community clients to pursue projects with, as defined in the Client Development section.

Results
Over the course of the 2017-2018 academic year, Med ECG worked with three clients on four projects. The clients included a local federally qualified health center, a local social services agency, and a metropolitan community health council. Projects covered topics such as operational efficiency, value-based health care contracting, and community health resource planning, and provided a variety of deliverables including automated reporting dashboards, board meeting recommendations, and summarized analysis. At the end of the project period, the entire group gathered for the project symposium, in which each team was able to present to the broader Med ECG group on the team's experience, specifically, the processes and analyses, recommendations, and key takeaways.
Next, we present the results for the three outcomes analyzed as described in the Methods section.
Outcome 1: Relevance to Existing Leadership Competencies Med ECG projects completed all five competencies of the MLCF (Figure 1). Students demonstrated personal qualities through learning to manage their limited time between coursework and their project obligations. They worked with others, including preclinical and clinical students as well as client partners,  and they managed both available resources and personal performance on all projects. While doing so, students engaged in critical, innovative thinking and problem solving and therefore completed the fourth competency (improving services) of the framework. Finally, the format and process of Med ECG projects mirrored the fifth competency of the MLCF (setting direction).
In the first few weeks of the project, each team was required to work with clients to scope the project and identify realistic contexts for change. Then, team members worked together to gather information and evidence to create informed, meaningful change within their client's organization. Ultimately, through final presentations and postproject surveys, team members had the opportunity to evaluate the impact their team created.
Med ECG's postproject reflection surveys satisfied Level 1 of Kirkpatrick's four-level training evaluation method. Moreover, client partners also evaluated project team skill levels across a variety of communication and professionalism attributes. However, because partner surveys were about the project team as a whole rather than individual team members and were unable to measure changes in skill level, they did not directly satisfy Kirkpatrick's higher levels of training evaluation. Therefore, Med ECG's training evaluation was defined as Level 1: reaction, corresponding to a change in learners' attitudes 18 (see Figure 1). Qualitative findings surrounding Med ECG's deliverables: r Responder 1: "As a result of Med ECG's support, we have an excellent dashboard to collect and report metrics and a great deal of the work to have our partners begin using the dashboard has already been completed. The dashboard is flexible and will allow for growth and adaptation as the project evolves." Impact of Med ECG project on client organization: r Responder 1: "Helped accelerate the progress of our project and produced a quality result that we will be able to use for the specific project. May be able to also adapt the dashboard for other projects as well." Surveys were also used to determine the effectiveness of project experiences on leadership, professional, and business skill development. Of the 14 respondents, 100% described their Med ECG project experience as valuable. Thirteen respondents (93%) said that they were planning to continue involvement in Med ECG, and the last noted "Maybe." The impact of project experiences on growth in particular attributes yielded a significant positive correlation between the amount of effort put into a specific attribute and the amount of improvement (R 2 = .61, p < .05). Figure 2 illustrates the relationship between these two variables (n = 140).

Discussion
Historically, medical education has focused on developing a competent clinical practitioner; however, the complexities of the modern health care system have gone beyond individual clinical practice. As medical students complete their training, they are entering a complex health care system in which they are expected to take on many responsibilities beyond caring for individual patients: leading teams, identifying and addressing system problems, controlling rising costs while improving quality, and many others. As a result, it is imperative that medical education maintains pace and provides trainees with opportunities to develop tangible leadership and business skills to prepare them for practice in this complex, team-based, interdisciplinary setting.
Project-based learning has proven to be an effective way to promote leadership development in a variety of extracurricular settings. [10][11][12][13]19 Our goal was to create a successful projectbased leadership program that provides medical students with real-world experience in solving problems for local nonprofit organizations. In its first year of existence, our 4-month program was both well received and effective. Med ECG's team members gained leadership, business, and professional skills by working on real projects with clients. The clients found the work to meaningfully impact their organizations. Last, Med ECG's activities fit established guidelines and frameworks for medical student leadership development.
Several challenges arose in the development of Med ECG. Clients were frequently slow to respond, at times limiting the pace at which projects could progress. We found that clients became more responsive after high-quality work was demonstrated and that careful consideration of clients' competing priorities, as well as upfront conversations about mutual expectations, was imperative for a successful working relationship. Our goal is to continue to develop ongoing relationships with this cohort of clients (and others) such that we can dependably form projects with responsive client counterparts. At the same time, medical student schedules were demanding and fluctuated throughout the year. To ensure equal student participation and prevent attrition, we informed students up front of the time commitment expected and created teams with students from different years in training such that team members' availabilities would complement each other.   Figure 2. Team member relative improvement versus effort for leadership and business skills. Analysis of the amount of effort put into a specific skill on the project and relative improvement of that skill during the project shows a strong positive correlation between the two variables. Therefore, the Medical Educational Consulting Group's consulting projects provided the most value to students when they were consistently working on a specific skill during the course of that project.
Additionally, project leads were responsible for individually distributing the workload such that projects would not interfere with medical coursework. This study has several limitations. Med ECG team members selfselected into the program based on their interests, providing a skewed sample of medical students who were more likely to be interested in leadership development than the average medical student. Second, with pre-and postintervention surveys, we introduced the possibility of response bias, in which respondents could have given more desirable answers, affecting the validity of the assessment. Nonetheless, this level of evaluation is commonplace for leadership curricula and helpful in understanding individuals' perceptions of improvement. Additionally, the number of responses from our team member and client surveys was constrained by our group's size, limiting the robustness of the conclusions. As Med ECG grows, we will be able to utilize larger sample sizes to better understand the relationship between projects and leadership development. Furthermore, all projects and surveys were completed in the first year after Med ECG's inception, and so, we cannot comment on longer-term outcomes in terms of leadership development or community impact.
In its current form as an extracurricular activity, Med ECG is sustainable as long as there are motivated students to take on client development and manage individual projects. In the future, we envision that interested students who have previously participated as team members can apply to be a project lead. We foresee that this role will be desirable, as it provides additional responsibility and opportunities to develop leadership skills. Analytical and Excel skill development demonstrate this apprenticeship model. As described within the Methods section above, the materials used to facilitate the trainings are transferrable such that someone versed in Excel can implement the materials at his or her institution. After learning Excel and analytical techniques through the training and their client projects, students who engaged as team members in their first year of participation should have the skills necessary to become project leads in subsequent years, teaching those skills to the next group of team members. The same principles apply to other leadership skills.
Additionally, although Med ECG is currently an extracurricular activity that students opt into based on interest, we believe that the program could easily be generalized and tailored as an elective within the medical school curriculum. Doing so would accomplish the following goals: (1) enable students to apply their work with Med ECG toward their graduation requirements; (2) secure ongoing faculty and administrative support for Med ECG projects; and (3) lay the groundwork for future interprofessional collaborations, including students in the schools of public health, nursing, pharmacy, business, and/or law. Generalizing the program such that all medical students could easily participate would further prepare medical trainees by equipping them with the leadership and business acumen necessary to thrive in the rapidly changing world of health care.