{'fraction_non_alphanumeric': 0.025305568130375735, 'fraction_numerical': 0.013444997736532368, 'mean_word_length': 5.3352451964439345, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Integrated vascular surgery applicants' perspectives of virtual residency interviews during the coronavirus disease 2019 application cycle

Background In agreement with Association of American Medical Colleges guidelines, the 2020 to 2021 integrated vascular surgery (I-VS) residency interviews were conducted virtually. In the present study, we collected data about the virtual interview process from the applicant's perspective, including preferences for interview format and the virtual resources they found most helpful. Methods An anonymous, online survey study of medical students who had been accepted into I-VS residencies during the 2020 to 2021 application cycle was performed. The survey contained questions about applicant behavior during the virtual application cycle, their perception of the virtual interviews, the utility of the virtual format, their preferences, and the virtual resources they found the most helpful for determining their rank list. Results Of 72 applicants, 38 (18 women, 19 men, and 1 declined to answer) had completed the survey for a 57.2% response rate. The average number of programs interviewed was 25 to 30 (31%). More than one-half (55%) of the respondents had responded that they had interviewed at more programs than they would have had the interviews been in person. More than one-half of the applicants (55%) reported that they preferred remote interviews with the option to visit in-person at their top choice programs compared with the use of all remote interviews (21%) or all in-person interviews (18%). Most had somewhat or strongly agreed (79%) that virtual interviews allowed them to properly gauge a residency program and provided adequate opportunities to interact with the residents (65%). The online resources the applicants found the most helpful in determining their rank lists were attending educational conferences, prerecorded videos from faculty and residents, and the program’s social media platforms. Conclusions The results from the present study have illuminated the current trends and attitudes of I-VS applicants for virtual interviews, including the virtual resources they found the most useful. Virtual interviews were a preferred method of interviewing and allowed applicants to assess a residency program and interact with the current residents. These variables should be considered by the program leadership when developing protocols for upcoming application cycles.

The coronavirus disease 2019 (COVID-19) pandemic affected all aspects of our healthcare system, including graduate medical education. Because of the social distancing guidelines, national organizations had encouraged programs to defer in-person interactions for the 2020 to 2021 residency application cycle. [1][2][3] As such, residency interviews were conducted virtually using video conferencing platforms.
The interview is one of the most important aspects of the residency application process. Both formal and informal interactions during the day and in-person dinners the night before have been cited as important for determining whether programs were considered a good "fit" from both the applicants' and the interviewers' perspectives. 4,5 Although virtual interviews have become increasingly popular and even the norm in other industries, they have remained rare in the healthcare field. Before the COVID-19 pandemic, little precedence was available on how best to conduct virtual interviews for residency or fellowship applications. Therefore, the drastic change in the interview format has resulted in questions regarding its validity, feasibility, and acceptability.
The use of a virtual format for residency interviews has significant advantages, including reduced costs for both applicants and programs and the creation of a more accessible process. 6 The increased cost of inperson interviews is primarily related to travel expenses (ie, airfare, gas, hotels). Although some institutions provided complimentary hotels, and most have provided meals, these practices were not uniform or consistent. In addition, travel expenses were not covered or reimbursed by programs. In addition, costs for food and other expenses will be accrued during the travel time.
Despite the cost-effectiveness of virtual interviews, applicants and programs were also concerned about the effectiveness of these changes, in particular because the interview provides critical opportunities for applicants and programs to evaluate one another for an ideal match. 7 Although the pandemic was the impetus for the change in interview format, the Association of American Medical Colleges (AAMC) has recommended continuing with the virtual format to improve equity and decrease the costs for applicants. 3 Although the pandemic limitations on travel have lessened, it is essential to understand the applicants' perspectives and trends toward this interview format to identify programmatic strengths and student needs and areas for improvement. Some data from other medical fields are available regarding medical students' and program directors' perspectives on the virtual interview process. 8 However, it is unclear whether these prior data would be generalizable to our field. Integrated vascular surgery (I-VS) residency applications are unique in the limited number of programs and positions available each year. Therefore, our research focused on I-VS applicants' experience during the first virtual interview cycle. The aim of the present study was to collect data about the virtual interview process from the applicants' perspective, including their preferences for the interview format and the virtual resources they found the most useful.

METHODS
In May 2021, an anonymous survey (Appendix, online only) was e-mailed to medical students who had been accepted into I-VS residencies for the 2020 to 2021 application cycle. The number of I-VS positions has increased from 66 in 2016 to 75 in 2020 and 79 in 2021. The medical students were identified from Society for Vascular Surgery's March newsletter of accepted applicants, the residency programs' online and social media platforms, and other public social media platforms. The applicants were encouraged to participate in the anonymous survey via the use of social media interactions, such as GroupMe groups. The anonymous online survey was e-mailed to 72 applicants. The e-mail addresses for eight accepted applicants could not be identified. In addition, one of us (M.C.) was an applicant during the 2020 to 2021 cycle, and her responses were included in the present analysis. The electronic questionnaire contained branching logic and a maximum of 17 questions and was administered via Survey Monkey. The survey contained questions about the applicant's behavior during the virtual application cycle, including their perceptions of virtual interviews, the utility of the virtual format, their preferences, and the virtual resources they had found the most helpful for determining their rank list. The institutional review board at the University at Buffalo exempted the present study.
Statistical analysis. The categorical data are reported as frequencies and percentages. Weighted averages and standard deviations were calculated for all ratings using Likert scales. Statistical analysis was performed using IBM SPSS Statistics (IBM Corp, Armonk, NY).

RESULTS
In 2021, 79 applicants were matched into I-VS residency programs. 5 We were able to distribute the survey to 72 applicants, 38 of whom had completed the survey, for a response rate of 52.7%. Of the 38 respondents, 18 were women, 19 were men, and 1 declined to answer (Table I).
ARTICLE HIGHLIGHTS d Type of Research: A cross-sectional survey study d Key Findings: More than one-half of accepted integrated vascular surgery applicants (55%) had preferred remote interviews with the option to visit in-person at their top choices compared with all remote interviews (21%) or all in-person interviews (18%). Also, most agreed (79%) that virtual interviews allowed them to properly gauge a residency program and provided adequate opportunities to interact with the residents (65%). d Take Home Message: The responses to our survey showed that virtual interviews were a preferred method of interviewing and allowed applicants to adequately assess a residency program and interact with current residents.
Interview behavior. The largest percentage of applicants had attended 25 to 30 interviews (31%; Fig 1). Most applicants (60.5%) had applied to both I-VS and general surgery programs. Most of the respondents (55%) also agreed that they had interviewed at more programs than they would have if the interviews had occurred inperson. In addition, 14% of the survey respondents had ranked a program in their top five that they would not have even interviewed at with in-person interviews. When asked to report their agreement on a Likert scale of 1 to 5 (1, strongly disagree; to 5, strongly agree), 79% of the respondents had strongly or somewhat agreed that remote interviews allowed them to properly gauge a residency program (weighted average, 3.57 6 2.2). When asked whether remote interviews provided adequate opportunities to interact with the current trainees, 69% strongly or somewhat agreed (weighted average, 3.48 6 3.2).
Interview preferences. When asked about their preferences, 21% of the respondents had exclusively preferred remote interviews, and 18% had exclusively preferred in-person interviews. However, most respondents (51%) preferred a combination, such that they could interview remotely but have the option to visit their top choices (Fig 2). Two respondents declined to answer the question about their interview preferences.
The applicants who reported preferring remote interviews valued the decreased cost of not having to travel (100%), the increased flexibility with scheduling dates (88%), and the ability to attend more interviews (63%; Fig 3). In contrast, those who had preferred in-person interviews prioritized their ability to directly observe the program culture (100%), visit the program's geographic location (86%), and eliminate the potential risk of technical difficulties affecting the interview interactions and influencing the applicant's and interviewer's impressions (57%; Fig 4).
Virtual away rotations and online resources. Ten respondents (26%) had participated in a virtual away rotation. Of these 10 respondents, 8 had ranked the program higher. When asked about the benefits of virtual away rotations, the applicants who had participated in a virtual away rotation commented on the increased interactions with faculty and trainees and the educational opportunities to learn more about the field of vascular surgery. However, many cited the lack of structure as a disadvantage to virtual away rotations.
The participants were also asked about various virtual resources that they had found helpful in determining their rank lists. The most frequently used virtual resources were grand rounds, the program's social media platforms, and virtual informational sessions. The resources ranked the most helpful were virtual educational conferences (weighted average, 4.13 6 2.7), virtual informational sessions (weighted average, 4.12 6 2.6), social media platforms (weighted average, 4 6 2.6), and virtual away rotations (weighted average, 4 6 3.1; Table II).

DISCUSSION
In the present study, we aimed to describe the applicants' perspectives regarding the virtual interview process during the COVID-19 pandemic. Although the pandemic is no longer the sole reason for the alteration in the interview process, the findings from the present study should be considered because they will continue to affect the ongoing remote interviews. The results of the present study have shown that most respondents had interviewed at 25 to 30 programs, and most had interviewed with more programs because of the virtual format. The number of interviews had increased from the previous application cycle, in which medical students who had matched into an I-VS program had interviewed in-person at a median of 17 contiguous programs.  The AAMC has, again, recommended strictly virtual interviews for the 2022 to 2023 interview cycles for all specialties, citing multiple issues, including equity and cost, as the primary reasons to continue the current remote interview process. The AAMC has also reported that after two successful cycles of remote interviews, they found that applicants had had more flexibility in scheduling and had missed less time from classes, in addition to realizing cost savings. 1,3 They have also recommended additional research to determine whether inequities might exist with the virtual process and to determine methods to mitigate those issues. 3 The increased number of attended interviews has raised some concerns by program faculty that applicants will attend interviews at more programs than they would consider ranking or attending thereby, possibly limiting spaces for other applicants who might be truly more interested in the program. 7,8 Although this is a valid concern, residency programs have also increased the number of applicants interviewed during the virtual cycle. According to the National Residency Match Program (NRMP) report on the virtual experience, one-third of programs had endorsed increasing the number of invitations extended and applicants interviewed and ranked. 9 Although these data included information from all residencies and were not specific to vascular surgery, the data suggest an increase in involvement from residency programs that mirrors the increased numbers seen in our sample. In addition, all I-VS residency positions were filled for the 2021 and 2022 application cycles, suggesting that this process was successful for the programs. 10,11 At present, no data are available regarding the "fit" of the trainees who had matched remotely vs. the "fit" of the trainees who had matched via the standard, in-person interview format.
Overall, the I-VS residency applicants embraced the virtual interview process. Most of our respondents selected it as their preferred method of interviewing. These   applicants valued the decreased cost and increased flexibility of virtual interviews. The financial burden of inperson interviews cannot be understated. A study from the previous application cycle showed that more than one-half of I-VS applicants had paid >$4000 in interview-related expenses. 12 Furthermore, one-third of the applicants had declined interviews because of financial restraints. 12 The use of virtual interviews will significantly reduce the cost of interviewing for both applicants and programs and can help create a more equitable application cycle for those experiencing socioeconomic barriers.
Despite valuing virtual interviews, the participants also emphasized the importance of visiting the location of the programs (18% preferred in-person interviews, and most preferred the option to visit their top-ranked programs). Engaging in group conversations or assessing interpersonal relationships through a computer screen can be difficult. Respondents also noted a concern about the potential for technical difficulties during their interview days affect the program's impression of them adversely. It can also be difficult to understand the hospital environment without visiting a program. If virtual interviews continue for future cycles, programs should seek to incorporate social events, the use of social media platforms, and activities that allow applicants to interact with current residents. They could also curate videos and websites that could better depict the geographic location of their program. Overall, these disadvantages did not seem to affect most respondents' preference for virtual interviews.
The recommendations for future cycles could include a hybrid approach, with interviews occurring virtually and applicants having the option to visit locations in-person, a process that was not available at that time of our study, nor has it been recommended, at present, by the AAMC. To create more equitable outcomes, students could be invited for a second look at programs after the program rank lists had already been submitted. This would require a change to the rank order submission date for programs by the NRMP and is not an option currently. Compared with other specialties, few I-VS programs are available, and the change could be coordinated in collaboration with the Association of Program Directors in Vascular Surgery, even without approval or sanction from the NRMP. Applicants can also visit the program's city and hospital outside the standard interview process. The concern with spontaneous, unstructured visits is whether this would bias programs toward applicants who had visited over those who had not visited or could not afford to visit.
Virtual electives had a positive effect on most of those who had participated during the COVID-19 pandemic, with 80% of participants ranking the site of the virtual rotation higher on their rank list. Applicants who had participated in a virtual interview stated that the strengths of the program included the opportunity to interact with the current faculty and residents. However, the limitations included poor organization with unclear didactic schedules and expectations. Furthermore, because in-person electives are again feasible, it is unclear whether virtual electives will be maintained and whether they will continue to have an effect on applicants' ranking of programs. Most applicants used online resources outside of the interview day to learn more about the programs. The most popular resources in our survey included virtual grand rounds, programs' social media platforms, and virtual informational sessions. These results highlight the importance of online resources for residency recruitment, regardless of the interview format. In the future, it will be necessary for program directors to continue to develop an accurate, up-to-date online presence for their programs. Some administrators might be less comfortable using social media platforms but should seek the assistance of technologically knowledgeable residents, program coordinators, and other faculty. Care is required to ensure that any online content development aligns with institutional social media policies. The findings from our study have highlighted the perspectives and trends that have not been previously studied in the current literature. However, the present study had several limitations. Our study was a single-year, cross-sectional survey study. In addition, our response rate was 52.7%. Therefore, a selection bias could have been present regarding those who had chosen to participate in the present study. Also, the survey was sent only to accepted applicants and might not reflect the preferences of the applicants who had not been successful in applying to I-VS programs. The survey was not e-mailed to those who had applied but had not matched owing to an inability to obtain an accurate list and the e-mail addresses of those individuals. Therefore, we could not evaluate the differences in the responses from applicants who had matched into an I-VS program vs those who had not matched. We also did not assess the effects of the virtual process on the faculty and programs to obtain their perspective.

CONCLUSIONS
To our knowledge, the present study is the first to investigate vascular surgery residency applicants' trends and attitudes during the first virtual interview cycle. Most applicants had interviewed at 25 to 30 programs and had accepted more interviews at more programs than they would have if attending in-person interviews. The virtual interview format was a preferred method of interviewing for the accepted applicants. However, most applicants would have preferred to interview virtually but maintain the option to visit their top choices in person. The virtual resources the applicants found most useful were virtual educational conferences, virtual informational sessions, programs' social media platforms, and virtual away rotations. These variables should be considered by the program leadership when developing protocols for upcoming application cycles. However, questions have arisen regarding the futility of virtual away rotations now that traveling bans and social distancing restrictions have been lifted. Additional research is necessary to assess the preferability and feasibility of a hybrid approach (remote interviews with the option for applicants to visit in person) for future interviews. All remote interviews All in-person interviews Remote interviews with the option to visit/interview for top choices Other (please specify) Q10. Why did you select the above answer (all remote interviews)?

AUTHOR CONTRIBUTIONS
Decreased cost due to absence of travel and hosting expenses Ability to attend more interviews Increased flexibility for interview times and dates Other (please specify) Q11. Why did you select the above answer (all in-person interviews)?
Opportunity to directly observe program culture Ability to visit the location Technical difficulties can affect the interview interaction and influence impressions Other (please specify) Q12. Why did you select the above answer?
Remote interviews with option to visit/interview for top choices (free response) Q13. Please rate how strongly you agree with the following comment: Virtual interviews allowed me to properly gauge a residency program Strongly agree Somewhat agree Neither agree or disagree Somewhat disagree Strongly disagree Q14. Please rate how strongly you agree with the following comment: Remote interviews provided adequate opportunities to interact with current trainees Strongly agree Somewhat agree Neither agree or disagree Somewhat disagree Strongly disagree Q15. How helpful were the following practices for gaining familiarity with a residency program (using the following rating: very helpful; somewhat helpful; neither helpful or unhelpful; somewhat unhelpful; strongly unhelpful; not available or did not participate): Virtual