{'fraction_non_alphanumeric': 0.03349723417332514, 'fraction_numerical': 0.026736324523663185, 'mean_word_length': 5.615853658536586, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reliability of adolescent standardised patients in assessing professionalism

Context and setting The Accreditation Council for Graduate Medical Education (ACGME) requires that residency programmes in the USA institute competency-based assessments of professionalism. Many residency programmes are struggling to find ways to assess residents’ competencies in professionalism. While standardised patients (SPs) have been used extensively in medical education, it is unclear whether adolescent SPs are reliable in assessing residents in this domain of competency. Why the idea was necessary Standardised patients are among ACGME’s recommended approaches for assessing residents’ competencies in professionalism. Usually, SPs are adult actors trained to portray realistic clinical scenarios. We have found that, in contrast with adult SPs, adolescent SPs are relatively easier to recruit and less costly. However, no prior studies have validated the reliability of adolescent SPs in assessing professionalism. If found to be reliable, adolescent SPs could offer paediatric educators a new tool for achieving ACGME competency mandates. Thus, we decided to examine adolescent SPs’ reliability in assessing residents’ professionalism. What was done We conducted a prospective criterion standard study at a major academic medical centre. Twelve adolescent SPs were trained to complete checklists that assessed elements of professionalism adapted from the ACGME Outcomes Project website. A cohort of residents (n 1⁄4 18) interviewed the SPs. All interviews were directly observed by 2 trained faculty members. The SP and faculty staff completed similar checklists immediately after each encounter and were blinded to each others’ ratings. Faculty ratings were used as the criterion standard. To validate SPs’ ratings, comparisons were made between faculty and SP ratings. Interrater reliability was assessed by determining percent agreement, Kendall tau-b coefficient of concordance and Cohen’s kappa coefficient of correlation. Evaluation of results and impact Overall, 36 resident– SP interactions were analysed. Results of interrater reliability analyses revealed statistically significant high levels of correlation between faculty and SP ratings in several domains, including trust, empathy, respect and integrity. Examples of statements from the checklists and their corresponding results of interrater reliability measures are as follows: ‘I can trust this doctor’ (percentage agreement 1⁄4 100%, Kendall tau-b coefficient 1⁄4 1Æ00, kappa 1⁄4 1Æ00, P < 0Æ0001); ‘The doctor showed warmth and compassion’ (percentage agreement 1⁄4 96%, Kendall tau-b coefficient 1⁄4 0Æ89, kappa 1⁄4 0Æ85, P < 0Æ0001); ‘The doctor was respectful to me’ (percentage agreement 1⁄4 100%, Kendall tau-b coefficient 1⁄4 1Æ00, kappa 1⁄4 0Æ90, P 1⁄4 0Æ001), and ‘The doctor asked for my opinion when making decisions about my treatment’ (percentage agreement 1⁄4 85%, Kendall tau-b coefficient 1⁄4 0Æ74, kappa 1⁄4 0Æ45, P 1⁄4 0Æ003). Results of this study provide further evidence in support of the use of adolescent SPs for the assessment of professionalism. We found that there was a strong correlation between faculty and adolescent SP ratings of the domains of professionalism.