{'fraction_non_alphanumeric': 0.019604705129231015, 'fraction_numerical': 0.003280787388973354, 'mean_word_length': 5.196331184928111, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Psychiatric-Specific Entrustable Professional Activity for the Evaluation of Prospective Psychiatric Residents: Towards a National Standard

Introduction Entrustable professional activities (EPAs) were developed as a way to ensure adequate skills of the medical school graduate. While the 12 EPAs apply to all medical specialties, EPA 1, “Gather a history and perform a physical examination,” applies most explicitly to psychiatry through the performance of a mental status exam. Although proficiency in performing a psychiatric interview and mental status exam evolves throughout a psychiatrist's professional life, basic proficiency is essential in order to function as a psychiatry intern. We developed a tool for assessing the mental status exams conducted by future psychiatry residents. Methods Our tool contains both a video of a psychiatrist interviewing a patient and a mental status exam rating sheet that can be used when students present a mental status exam orally or in writing. We incorporated feedback from psychiatry educators at an annual meeting of the Association for Medical Student Educators in Psychiatry, followed by the reiteration of the video and the rubric. Subsequently, the rubric was verified on the performance of a cohort of 13 third- and fourth-year medical students from three institutions. Results In their mental status exam presentations, students covered all the items measured by the rubric. There was a significant difference between the third- and fourth-year medical students in describing the cognitive exam. Discussion Overall, our tool offers an opportunity to standardize mental status presentations by senior medical students who wish to specialize in psychiatry.


Introduction
A competency-based curriculum in contemporary medical education organizes the educational experience around skill proficiency, underscores performance outcomes, and ultimately promises greater accountability to patients and society. Graduate medical education accreditation agencies historically have required educational programs to implement competency-based evaluations. The Accreditation Council for Graduate Medical Education has developed a matrix of milestones corresponding to each developmental stage in specialty training. In 2014, the Association of American Medical Colleges followed

Original Publication
suit with the development of entrustable professional activities (EPAs) as a way to ensure that the medical school graduate possesses an adequate set of competencies on the first day of residency.
Although proficiency in performing a psychiatric interview and mental status exam evolves throughout a psychiatrist's entire career, basic proficiency is essential for a new medical graduate entering into a psychiatry residency program. Though medical students are often assessed on their ability to perform a mental status exam both clinically and on observed structured clinical examinations during their psychiatric clerkship, this skill often takes dozens of repetitions to perform well. As such, medical students expecting to pursue a career in psychiatry are expected to continue to refine this skill throughout the rest of their clinical years before being adjudicated as having a full EPA and therefore being prepared for residency.
Here, we depict the development of an assessment tool for a psychiatry-specific EPA to be used with senior medical students pursuing psychiatric residencies, particularly regarding their ability to identify, describe, and document abnormal physical exam findings as part of EPA 1 (i.e., gathering a history and performing a physical examination). In the case of psychiatry, this may refer to the mental status exam.
The mental status exam is the primary (and often only) portion of the physical examination that all psychiatrists use with each patient encounter. The mental status exam is often abbreviated in other disciplines, whereas in psychiatry, it is used in its whole form.
Often, the last assessment of a student's skill occurs at the end of his or her clerkship. This does not allow for the opportunity to see further growth of this skill throughout the remainder of the student's education.
Our tool can be used in a capstone assessment. Whereas the clerkship-level education and evaluation should be sufficient for the graduating student who is not going into psychiatry, they may not constitute an EPA for a future psychiatric resident. Therefore, this tool may be used to assure that the future psychiatrist meets EPA 1 for psychiatry.
The ability to perform and report a complete mental status examination is a unique EPA for future psychiatrists. Although resources that teach the psychiatric interview with the mental status component are available on MedEdPORTAL, our resource is unique in that it specifically addresses the mental status exam and the systematic evaluation of its presentation. It prepares the trainee for higher-stakes performance evaluations such as the United States Medical Licensing Examination Step 2 Clinical Skills exam and ultimately for the basic psychiatric residency milestone of Psychiatric Formulation and Differential Diagnosis, which states that a competent psychiatric resident "organizes, summarizes, reports, and presents to colleagues the information obtained from the patient evaluations." This tool allows for a standardization of the interview, which leads to a specific expectation on the mental status exam, and reduces the time burden on evaluator who would otherwise have to find a patient, observe the learner's evaluation, and then review the mental status exam.

Methods
This mental status exam rating tool comprises a video recording of a psychiatric interview and a scoring rubric with anchors describing levels of competency in presenting a mental status exam. The video (Appendix A) was filmed by one the authors (Benjamin T. Griffeth), utilizing a script adapted from Klapheke. With Dr. Klapheke's endorsement, his script was expanded to focus on the cognitive examination.
The scoring rubric, referred to as the mental status exam rating sheet (Appendix B), was developed by one includes an evaluation on the following items: appearance, behavior, psychomotor activity, speech, affect, mood, thought process, thought content, perception, insight, judgment, Montreal Cognitive Assessment (MoCA) global score, and clarity of presentation. Each item is measured as either a pass, minimal pass, or fail. An overall score is also assigned as a pass or fail. An item failure is assigned when a student fails to address a section or provides a response that is abjectly incorrect (e.g., identifying a patient's smiling and open affect as depressed). A passing score is awarded when the answer is consistent with the expectation of a PGY1 resident (e.g., the correct identification of pressured speech in a patient presenting with rapid speech that is difficult to understand or difficult to interrupt). A minimal pass score is assigned for close-tocorrect descriptions (e.g., if the student describes a patient with pressured speech as having fast speech).
An overall failure score may stem from incorrect presentation of data (e.g., a report of normal speech in a patient with pressured speech) or failure to address the criteria at all (e.g., no report of the speech of the patient). three fourth-year students who were choosing psychiatry as their intended residency and nine third-year students who had yet to make a choice of intended residency but who had completed the psychiatric clerkship watched the standardized video of a psychiatrist performing a limited examination on a patient and were allowed to take notes during the viewing. Each student wrote out a complete mental status examination with no time limit. The students were not aware of the rubric content and described the mental status exam exclusively based on the teaching they had received in the clerkship. The clerkship faculty at each of the three sites then graded the students' work using the peer-validated rubric.

Results
All 12 students passed the rubric as a whole. From 169 grading points (12 students each graded on a 13item rubric), students received full passing grades for 153 out of 169 (90.5%). Of the 12 students, only five had more than one minimal pass, and only one posted a failure of any component. The cognitive exam component received the most minimal pass responses as well as the only item failure (missing the MoCA score by more than 2 points and not addressing the areas of deficit). Comparison of the nine third-year students to the three fourth-year students revealed that only the ability to document the cognitive exam was different between third-year and fourth-year students (i.e., one out of nine third-year students passing vs. all three fourth-year students passing). This difference was found to be significant using the Fisher exact test. Due to the small number of participants in each group, we used this test to compare categorical 7,8 data as a more appropriate statistical method. A comparison of item scores across the two student groups may be seen in the Table.

Discussion
As medical education moves toward increasingly competency-based assessments, this module offers the opportunity to have a nationally available, validated assessment tool for EPA 1, particularly to "identify, describe, and document abnormal mental status exam findings." Though clerkship is the only formal education in psychiatry at most schools, ongoing exposure to psychiatric patients and psychiatric tools throughout the clinical years gives graduating students a more robust education than what is received in the psychiatry clerkship alone. Although our initial sample of students is small, there is evidence of content validity given the fact that we were able to score all 13 items on the rubric based on the responses provided by students.
Explanations as to why the fourth-year medical students performed better on the rubric than the third-year students validate our expectation that broader clinical exposure is additive to the clerkship's teaching and includes such benefits as greater patient contact and greater exposure to the mental status exam. Additional sources of variance between the third-year and fourth-year students may center on the greater interest of fourth-year students intending to go to psychiatry residency versus third-year students who are as yet undecided or have opted to not go into psychiatry. The ability to show variance between the groups through this evaluation tool allows clerkship directors to validate that fourth-year medical students seeking psychiatric residency are able to demonstrate EPA 1 whereas those who have completed only the third year of medical school psychiatric rotation are not able to demonstrate EPA 1.
There are a small number of limitations with this resource. The number of students in the pilot was small, although their robust performance indicates the strength of the tool and their preparation. Additionally, student performance could not be measured against another assessment of a mental status examination as the comparison test would not have been similar between the three testing sites. However, having a standard video to present to students, as well as a standardized rubric, brings objectivity to the examination and will allow comparisons to be made between different sites and campuses at the same medical schools listed previously. Additionally, with a vast majority of students successfully completing most of the sections appropriately, the current tool may be limited in its ability to discriminate between those fully developed in the EPA and those who are progressing. Another limitation is that the rubric gives subheading and examples for potential responses of the student to the various portions of the mental status exam but does not indicate correct answers or differential responses that would be acceptable. Future rubrics would benefit from removal of examples that are not germane, such as hypnogogic hallucinations and jealous delusions, as well as inclusion of stronger descriptions of the expected definitions of insight and judgment.
Faculty will benefit from this tool by being able to identify specific areas of remediation that may need to be developed before the future resident can be said to have a complete EPA. In the future, we plan to engage medical schools from other regions in the United States for comparisons of student performance.