{'fraction_non_alphanumeric': 0.023293768545994065, 'fraction_numerical': 0.01172106824925816, 'mean_word_length': 5.2294362292051755, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Initial Experience with a Virtual Platform for Advanced Gastrointestinal Minimally Invasive Surgery Fellowship Interviews

Background The COVID-19 pandemic travel restrictions triggered a rapid alteration in the interview process for fellowships this spring. We describe our initial experience with virtual interviews for Advanced Gastrointestinal (GI) Minimally Invasive Surgery Fellowships and assess the value and limitations via a post-interview applicant survey. Study Design Twenty candidates were interviewed via Zoom teleconferencing during March and April 2020 using combined group and breakout rooms. An anonymous post-interview Likert and free text survey was sent to candidates with questions regarding feasibility, appropriateness, and acceptability of this method. Results Seventeen of 20 candidates (85%) responded to the survey. The candidates rated ease of interaction with the program director, faculty surgeons, and the current fellow highly: 94%, 83%, and 89%, respectively. The majority (53%) stated the virtual interviews exceeded or met expectations. Only a minority, 12%, reported the virtual platform was short of expectations. Approximately 70% noted little to no impact of not being able to conduct these interviews in-person and not being able to physically see the program institution. Overall, 94% were satisfied with their experience, and only 6% were neutral, with no respondents reporting dissatisfaction. Finally, 76% would recommend a virtual interview in the future. Most negative open response comments were secondary to issues with software rather than the lack of the in-person traditional interviews. Conclusions The use of a remote teleconferencing platform provides a favorable method for conducting fellowship interviews and results in a high degree of candidate satisfaction. Virtual interviews will likely be increasingly substituted for in-person interviews across the spectrum of medical training.

Remote video teleconferencing has been a part of modern business work culture for many years. Internetbased real-time communications have provided a robust alternative to in-person meetings beginning in the 1990s with the advent of web-based instant messaging. A steady rise in use of video teleconferencing has occurred in the last 2 years, in part due to improvements in internet bandwidth along with more capable and available computing devices. 1 Despite advantages of feasibility and reduced costs, medical schools, residency, and fellowship programs have been slow to adopt this technology because of the perceived numerous benefits of in-person face-to-face interviews and visits. Due to the rapidly growing 2019 coronavirus pandemic (COVID-19)-related travel restrictions and need for social distancing measures, medical institutions have implemented remote video teleconferencing in a very short timeframe that has essentially replaced in-person meetings at all levels, from educational conferences to committee/staff meetings, to candidate interviews. 2 Teleconferencing has observable benefits, primarily with reduction of travel costs 1 associated with inperson meetings and with increased flexibility in timing for 1 or multiple parties, which may improve the connectedness and effectiveness of remote teams. 1,3 Despite these clear benefits, concerns arise with use of a remote platform, both for the interview process itself and for potential consequences for candidates and institutional training programs. Specifically, the lack of face-to-face personal connection during the interview of a potential candidate may obscure or limit impressions from nonverbal cues or obfuscate intonation or intent of speech. Certain elements, such as eye contact, poise, or a sense of "fit," which are revealed during inperson interviews, may provide insights that are not easily replicated on a remote platform. In parallel, technological hurdles with each software platform, along with connectivity issues that arise with any web-based interface, can further degrade the process, experience, and impressions on both sides. 4 Despite the potential downsides, use of this technology for the interview process, due to the COVID-19 pandemic, has become widespread.
Fellowship training after general surgery residency has become the norm, with studies showing greater than 80% of trainees seeking fellowship positions. 5,6 The interview process can lead to significant personal cost, both by creating substantial missed time, which can negatively affect a trainee's clinical experience, and the cost of travel for multiple interviews further adding to a trainee's debt. 5,7 In light of this, the use of remote conferencing may provide a beneficial alternative or adjunct to the current structure of travel-based, in-person fellowship interviews. Because there are no reports to date that have examined a total transition to virtual interviews for post-residency fellowship training in surgery, the objective of this study was to describe the experience of fellow applicants with the interview process and assess the feasibility, appropriateness, and acceptability of using a single remote video teleconferencing software platform during the 2019e2020 Advanced Gastrointestinal (GI)/Minimally Invasive Surgery (MIS) fellowship interview cycle.

METHODS
The Fellowship Council, which oversees the application and match process for several non-ACGME fellowship types, which includes all Advanced GI/MIS fellowships, issued an advisory on March 17, 2020 that all nonlocal fellowship interviews should be conducted via an alternative method due to the COVID-19 pandemic and related travel restrictions. As a result, the 3 interview dates previously scheduled for this application cycle by our group (2 at our local institution and 1 during the Society of American Gastrointestinal and Endoscopic Surgeons annual meeting) were transitioned to a virtual format. Applicants were contacted and rescheduled for the same dates as previously organized, with some modifications in the schedule, using the Zoom platform (Zoom Video Communications, Inc). Fellow candidates were sent a revised itinerary via email with a link to the Zoom conference for their interview day, which was held between March 27 and April 10, 2020. Institutional IRB exemption for the study was obtained.
On the interview day, fellowship applicants, faculty, the current MIS fellow, and the program administrator were joined in a central hub virtual meeting room. The Zoom platform was structured such that from the combined central hub, applicants would be placed into breakout rooms, where they would have a 15-minute interview with a faculty member, and then both applicants and faculty would return to the central hub before being reassigned to the next breakout interview (Fig. 1). A trial run of the hub/breakout room set up was carried out with 3 faculty in advance of the interview day. The interview day began with a presentation overview of the MIS fellowship that was the same as in previous years with on-site interviews. The candidates were next given a brief orientation to the Zoom platform and use of the breakout rooms. Each candidate and faculty member were then placed into a breakout room for 15-minute 1-on-1 interviews with a timer running continuously in the upper right corner of the screen. An alert appeared when 1 minute was left in the session. Any candidates not currently in an interview (when there were more applicants than faculty) remained in the central hub room with the current MIS fellow for informal questions about the program and city. The breakout rooms were assigned and then reassigned during a brief break between interviews and took, on average, less than 1 minute to complete. This was done by an interviewing surgeon on the first day and by our program coordinator for the second and third interview days.
The duration of the interviews was the same as for our originally scheduled in-person interview this year and in previous years. The interviews were not formally structured, but faculty were encouraged to use 1 rotating standardized question per interview. At the end of the interview process, the candidates remained in the hub room with the current MIS fellow for a brief presentation and final question and answer session. Fellowship candidates who had conflicting arrangements were encouraged to leave if necessary and were provided further contact information for the faculty, staff, and the current MIS fellow to address any remaining questions.
After the interviews, all candidates were sent an anonymous voluntary survey (Table 1) to assess their impressions of the remote teleconferenced interview process through QualtricsXM (Qualtrics, LLC). Primary categories of assessment were feasibility, appropriateness, and acceptability of this platform along with information about attitudes and previous experience with remote interviews and finally prompts for comments. A similar survey was sent to all interviewing faculty members to assess their perceptions of the virtual interview experience. The results were tabulated and reported. IRB exemption was obtained for this study.

RESULTS
Twenty applicants were invited for in-person interviews from a total of 110 fellowship applicants. The candidates who interviewed were from 17 different institutions and were located in Eastern, Central (same as host program) and Pacific time zones. Nine candidates were women (45%) and 11 were men (55%). Three applicants who resided locally were also interviewed virtually in order to maintain consistency and fairness of the interview process. The number of candidates interviewed on a given day ranged from 3 to 9 individuals. For the 2 largest sessions (8 and 9 candidates), applicants were interviewed by all 7 faculty; for the session with 3 candidates, interviews were conducted by 5 faculty. Only 1 candidate moved from the originally scheduled date to a different one. Due to the complete shutdown of all elective clinical activity, 2 faculty were located in their academic offices and the remainder were at home. Complete post-interview survey responses were received from 17 candidates (85% response rate). Results from the applicant survey are shown in Table 2.

Feasibility
Most responses indicated that interacting with the fellowship program director, individual surgeons, and MIS fellow with the provided platform was easy (94%, 83%, and 65%, respectively) and was extremely easy in 59% to 65%. One candidate (6%) responded neutrally to the platform for these interactions, and 2 candidates (12%) reported some difficulty with interacting with the faculty and MIS fellow only.

Appropriateness
The overall experience of the fellow candidate was rated as meeting or exceeding expectations in 88% of responses. The remaining 12% stated that the remote interview  How did NOT physically seeing the hospital and city affect your opinion of the fellowship program? (1) Yes, (2) No How many prior video conference interviews had you been on prior to this interview?
Open ended What are the weaknesses of the virtual interview process?
Open ended MIS, minimally invasive surgery. The inability to physically see the medical center, city, and program/institution was rated similarly, with 70% stating there was little to no impact compared with 30%, who rated this aspect as having a moderate to great deal of impact.

Acceptability
The vast majority of candidates, 94%, reported being satisfied with the overall experience of the remote interview. All candidates reported some degree of benefit from the warning near the end of each session, though the relative level of usefulness was quite variable, with 70% rating the prompt as extremely or very useful vs 18% who found it only slightly useful. Only 1 candidate reported no benefit from the orientation session. Finally, the number of prompts were rated as manageable to great in 100% cases, with the most common response in the manageable category (65%).

Attitudes
Applicants rated their overall attitude toward the remote platform before this interview as positive in 47% of responses, neutral in 40%, and somewhat negative in 13%. It should be noted that 15 of the 17 respondents answered this individual question. The majority (76%), however, would recommend this platform in the future for remote interviews compared with 24% who would not. More than half of the applicants (53%) had already been through 5 or more remote interviews, and another 27% had been through 2 to 4 previous in-person interviews. Only 1 interviewee had no previous remote interview experience. Overall, 76% of candidates rated this remote interview experience as better (somewhat better in 69%) than their previous ones, with no respondents rating it comparatively worse.

Strengths/weaknesses
Nine of the 17 applicants (53%) provided subjective feedback text comments on the process. Of those who responded subjectively, 89% directly mentioned cost savings as a strength of the interview, and 45% mentioned reduction in missed time and improvement in flexibility of scheduling. Regarding weaknesses, 89% mentioned the lack of the in-person aspect of both seeing the city and faculty as a drawback. Three applicants (33%) specifically mentioned technical issues and 2 mentioned that the 1-on-1 interview session being cut off/timed out by the software was detrimental.

Faculty assessment
Faculty survey results are shown in Table 3. All 7 faculty (100%) rated the ability to interact with the candidates as extremely easy. Five faculty (71%) stated the virtual interview experience exceeded their expectations, with the remaining 2 (29%) stating that it far exceeded their expectations, with no faculty members reporting neutral or negative responses. Regarding the virtual interview affecting the traditional in-person interaction, only 1 (14%) reported a moderate effect, while the remainder reported little (57%) to no effect (29%). All faculty were satisfied, with the vast majority (86%) being extremely satisfied with the interview day. This was reflected with 100% of faculty members recommending continued use of a virtual platform for fellowship interviews for MIS.

DISCUSSION
Of the many adaptations that have taken place in the last 2 months due to the COVID-19 pandemic, 1 major change that will likely endure is the adoption of remote teleconferencing across various facets of life, both personal and professional, as an alternative for many traditional inperson interactions. The use of remote teleconferencing for faculty, staff, and committee meetings, patientphysician visits, educational sessions, and even job interviews has risen sharply across multiple professions, including medicine. 4,[8][9][10][11][12] In fact, the ACGME has recommended that all medical student interviews for the 2020-2021 residency cycle be performed virtually. 13 Multiple institutions have already shown the feasibility of using telemedicine to implement and expand clinic visits in the short time that the pandemic has been crossing the US. 8,9 Given the widespread travel restrictions that were implemented in March 2020, the spring Advanced GI/ MIS fellowship interview process was disrupted midcycle, requiring that programs conduct their subsequent interviews remotely. Because this process is new to almost all training program interview processes, we sought to describe how our sessions were constructed, which may serve as a model for others going forward. We also used a post-interview survey to assess the feasibility, appropriateness, and acceptability of this with largely favorable experience. Our findings demonstrate that despite concerns about the lack of in-person interaction, the majority of experiences reported by applicants were positive regarding the ease of interaction with both faculty and the current MIS fellow, were generally satisfied with the overall interview process, and would recommend this platform for continued use. Many platforms and strategies exist for how to construct a virtual "interview day" experience when multiple candidates are involved. In considering a virtual platform for a traditionally in-person event, the ease of interaction is paramount. It is also critical that the individuals be able to see each other, and ideally see the entire group to facilitate team interactions with the candidate. The stability and clarity of the software and internet connection are also important factors to consider. To ensure this, the hosts for our interviews were based in an academic office using a university-based server. Because our university information technology (IT) service established Zoom as the primary institutional virtual platform 2 months before the COVID-19 disruptions, we chose to use this approach for our interview sessions. Rather than physically rotating faculty around different offices with virtual connections already made, we took advantage of the breakout room feature in Zoom. One challenge was completing a conversation thread before the session automatically ended and bumped everyone back to the central hub. This happened on several occasions, whereas with in-person interviews, there is always the option to run beyond the scheduled time limit and finish the discussion. Despite this limitation, the process worked well, as evidenced by the survey results, and this issue could potentially be managed in the future by alternative settings on the Zoom system.
Another aspect of video interviews that must be considered is the security of the event. There have been several reported instances of "Zoom-bombing" that have disrupted virtual meetings. Security precautions, which should be taken to avoid this, include defaulting screen sharing to the host only, requiring a meeting password, leaving personal meeting identification to default off, avoiding the sharing meeting information on public websites or social media, using a waiting room for admission to the meeting, and locking the meeting after it begins. 14 Our data suggest that the ease of interaction with all members of the interview day were rated quite positively. Almost all applicants (>80%) felt that it was extremely easy to interact with all parties, with only a small minority reporting difficulty. Of note, a recent survey of video conferencing found 40% of meetings have some form of hardware or software issues, and only 17% of those surveyed were satisfied with their experience. 15 Although observationally it appeared most candidates had a similar experience during the interview day, a few candidates did have issues with connectivity and the logistics of the software that were present mostly in the first interview day. One applicant had issues with connecting to the central hub room and another had difficulty with the audio feed, both of which were resolved during the session. During the first round of interviews, the timer was an issue in the first iterations of the breakout rooms, in that it would abruptly cut off the session without warning. This was resolved quickly with further experience with the software. In the subsequent sessions, the majority of these difficulties were addressed before starting, and there were minimal technical issues after the learning curve from the first interview session. We suspect that some connectivity issues are inherent to a remote platform, but the growing pains of acclimating to new software are short lived. One unexpected occurrence was that during the program overview part of one interview day, the building fire alarm went off for a period of more than 5 minutes, which interfered with the audio communication and necessitated transferring the screen control and the remainder of the presentation over to another faculty member who was located remotely. This issue could be avoided in the future perhaps by use of an earphone/ microphone system to minimize the distraction from ambient noise.
One major concern of the faculty has been the lack of in-person interaction, and that the inability to see the hospital and city may adversely affect the applicants' impressions. Our data suggest that this may play a lesser role than initially hypothesized, with approximately 70% of candidates and 86% of faculty suggesting little to no effect of these limitations It appears that candidates placed greater importance on the in-person aspect of interviews rather than the tour or experience of the city, as evidenced by 7 candidates (41%) reporting no effect of not seeing the program, whereas only 2 candidates (12%) reported no effect of the lack of in-person aspect of the interview. In review of the subjective comments, 8 applicants did specifically mention that one of the drawbacks of the virtual process was the lack of seeing the city and the facilities. However, only 1 of the 8 applicants who mentioned this issue in the written comments also rated that it ultimately had a large effect on the overall experience. Perhaps this discrepancy is due to other factors, such as preconceived impressions of both the program and city, reputation of the program, or simply that the particular setting of a fellowship program is less important than the training experience offered. Also, having 3 local applicants may have diminished any negative responses on this aspect of the experience, leading to potentially biased results. Although our survey was not equipped to answer this question, this finding may serve to lessen the emphasis traditionally placed on the physical visit to a fellowship program.
A major consideration of the trainee interview process is the cost to the individual, especially when multiple fellowship programs are being visited in multiple different cities. Given the significant travel expenses associated with fellowship interviews, with evidence that most residents/ fellow candidates spend $4,000 to $6,000 and miss 7 or more days of work during the entire interview season, alternatives to the existing process are desirable. 5,10 The inperson interviews also add cost to the program, which for our program, has ranged from $1,800 to $2,000 for each interview season. Watson and colleagues 5 found that 37% of fellowship applicants favored a central location for interviews to save on the high cost of travel, and more than 32% reported the need to borrow money for the interviewing process. 5 In contrast, approximately 70% of the fellowship program directors themselves favored changing the fellowship interview process to a central location. It is unclear why this paradox exists. Our data suggest than when obligated to go through an alternative, virtual centralized process, the majority of fellowship candidates actually viewed the experience positively, with fewer shortcomings than expected.
Nevertheless, there is still a sizeable contingent of residents who place importance on the in-person aspect of the interview and seeing the city and the program. One mechanism to resolve these conflicting aspects would be to hold a remote preliminary screening interview of candidates with a second in-person visit to a program for a smaller sub-selected group of candidates. Our data demonstrate that despite the unexpected change to the traditional process, the majority of applications and all of the faculty in our department would recommend continued use of a virtual platform for interviews in some capacity. The idea of a tiered interview process has been proposed before and may prove to be an optimal solution, with use of a remote platform as an adjunct. 16 Furthermore, use of prescreening questionnaires to help select those who may be offered an interview has been shown to reduce the number of applicants who ultimately need an in-person interview. 17,18 Although borne out of necessity, the virtual interview platform will likely spur a paradigm shift from the current process.
Our study has several limitations. Chiefly, there was no comparative control in-person interview group. Additionally, pursuant to the number of candidates interviewed by our program, the sample size is small, and we did not ask candidates to compare this experience with in-person interviews they had earlier in the interview cycle. Opinions regarding the 3 queried categories could be influenced by each candidate's previous interview experiences, which introduces some variability into the final assessment. Even though the survey was anonymous, some applicants could have been subconsciously influenced to provide favorable responses due to prior knowledge of the program, program faculty, or other factors. In fact, 3 local applicants (15%) were already quite knowledgeable about most aspects of the training experience, and their opinions, therefore, may have been less likely to have been influenced by the virtual format. Finally, the use of the Zoom platform as the sole method undoubtedly creates both positive and negative technical aspects that could be different if another software platform were to be used, and so the reported experiences may not be generalized to the feasibility of remote teleconferencing.
Our interview day differed from those in previous years in some aspects: the absence of the weekly case conference, breakfast at the start of the day, a tour of the medical center, and lunch to conclude the day. In the future, the case conference could be easily incorporated into the virtual experience, and a video medical center tour is something that has broad appeal across multiple departments, as forthcoming fall and winter interview cycles will likely be disrupted as well. Despite these limitations, our findings provide positive evidence that remote interviews may be a useful adjunct if not a complete replacement of the traditional process. It is our vision for the future after COVID that we will use the virtual platform for screening interviews of candidates, which would likely result in fewer candidates who would be offered an on-site interview, at reduced expense for candidates and less travel time away from residency. We would also use virtual interviews for those invited, but unable to attend in-person, and as a preferable alternative to interviews at a national meeting where scheduling and time constraints are limiting.

CONCLUSIONS
The COVID-19 era has forced many new adaptations to ensure a more physically distanced society. In developing an alternative process to traditional in-person fellowship interviews, albeit for one fellowship type, we have shown that remote teleconferencing provides a feasible platform to conduct interviews that is rated favorably by applicants. Given that the implications of the COVID-19 pandemic on travel and group interactions may last for the next many months, this format will likely be used for residency and fellowship interviews nationally in the coming year(s). Our initial venture suggests that this approach is readily adopted and may provide an experience that is comparable in most respects to the traditional interview setting.