{'fraction_non_alphanumeric': 0.04812286689419795, 'fraction_numerical': 0.025938566552901023, 'mean_word_length': 5.316810344827586, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Applicant Assessment During Virtual Interview as Compared to In-Person Ratings

Rationale: Pulmonary and Critical Care Medicine (PCCM) Fellowship interviews transitioned to virtual format 2020 due to the COVID-19 pandemic, yet the impact of virtual interviewing on interviewer assessments remains unknown. Virtual interviewing has the potential to negatively impact interviewer perceptions of interpersonal skills given the challenges of video-based conferencing. We aimed to assess differences in quantitative ratings across four interview domains for in-person compared to virtual interviews, as well as for sentiment differences in accompanying narrative comments. Methods: We performed a single-center retrospective cohort study of all interview assessments completed for the University of Pennsylvania PCCM fellowship program for the 2019 interview season (in-person) and 2020 interview season (virtual). Interview assessments consisted of four discrete domains (leadership, interpersonal and communication skills [ICS], scholarship, and clinical skills) on a 5-point rating scale, submitted via an online evaluation. Each numeric rating was accompanied by optional free-text comments, in addition to a free-text field for overall impressions. Summary statistics of ratings were calculated for each domain, and t-tests (p 0.05) were used to assess for statistically significant differences between years. Free-text comments were assessed using Linguistic Inquiry and Word Count (LIWC) analysis, which calculates the degree to which various categories of words are used in a text and quantifies the overall sentiment of text. Results: 104 unique residents where included in the cohort, with 52 in each season. The cohort consisted of 52% (n=54) women. Fifteen-percent (n=16) of candidates identified as under-represented in medicine. (Table) Overall mean ratings in each of the four domains (leadership, ICS, scholarship, and clinical skills) were 3.76 (+/-0.79), 3.55 (+/-1.09), 3.69 (+/-0.89), and 3.97 (+/-0.76), respectively. There were no significant differences between in-person and virtual formats in mean ratings for clinical skills (p=0.25), leadership (p=0.73), or scholarship (p=0.48). However, the mean ICS rating was significantly higher during virtual interviews compared to in-person interviews (3.96 versus 3.15, p < 0.001). There was no significant difference in LIWC sentiment analysis rating in the free-text domains, including for comments in the ICS domain. Conclusions: Virtual interviews led to higher mean ratings in ICS, compared to in-person interviews, with no text-based differences in narrative comments. These differences were not demonstrated in other domains, including leadership, clinical skills, and scholarship. This finding was unexpected, especially given the decreased proportion of women applicants in 2020, and may suggest leniency bias in ratings of ICS in virtual interview formats.