{'fraction_non_alphanumeric': 0.025748202641698713, 'fraction_numerical': 0.013710081926099314, 'mean_word_length': 5.407970859224341, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Rapid Validation of Telepathology by an Academic Neuropathology Practice During the COVID-19 Pandemic

This article was posted on the Archives Web site as an Early Online Release. Note: Due to the extremely time sensitive nature of the content of this article, it has not been copyedited or formatted per journal style. Changes or corrections may be made to this article when it appears in a future print issue of the Archives. Early Online Release articles are citable by using the Digital Object Identifier (DOI), a unique number given to every article.


INTRODUCTION
On December 29, 2019 official reports describing a cluster of patients suffering from a severe novel pneumonia of unknown etiology initially surfaced from Wuhan (Hubei province) in the People's Republic of China. [1][2][3][4][5][6][7] It is hypothesized that the illness, coronavirus disease 19 , is likely the result of an initial zoonotic event that may have occurred in the Huanan seafood wholesale wet market. [6][7][8][9] On January 7, 2020, the causative infectious agent, severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) was officially described. 2,5,7,[10][11][12] COVID-19 has a reported incubation period of roughly 2-14 days, following which patients may present with fever, dry cough and/or shortness of breath. 10 While a significant subset of individuals can be asymptomatic or only mildly affected, others, particularly the elderly and those with pre-existing conditions, may exhibit a moderate to severe illness, which may require mechanical ventilation and can be associated with an increased mortality risk. 4,[11][12][13] In January 20, 2020, western Washington became the first region in North America with a confirmed COVID-19 case. [12][13][14] Subsequently, Washington state became the first major site of the COVID-19 outbreak in the United States of America. 11,[15][16][17] On March 11, 2020 the World Health Organization (WHO) declared COVID-19 a pandemic; one that as of June 1 st , had reported over 6,363,000 positive cases and at least 377,000 deaths across 215 countries and territories including over 1,859,000 positive cases and at least 106,000 deaths in the United States of America. 18 The nature of the work and interactions occurring in anatomic pathology laboratories, together with viral features such as asymptomatic carrier shedding and prolonged stability in aerosols and surfaces can result in SARS-CoV-2 transmission events even in these non-direct patient care areas. [19][20][21][22][23][24][25][26] Workplace transmission events may lead to reductions of laboratory personnel and/or physician staff due to individuals need to self-quarantine due to exposure and/or sickness. This in addition to other unknowns following infection, including whether or not immunity is conferred or if there is a potential for relapse and/or reinfection of self or others following initial sickness resolution, may further complicate these workplace transmission events. 27,28 Age is another complicating factor; a study conducted over a decade ago reported that 75% of the pathologist workforce in the USA was 45 or older, with 12% being 65 or older. 10,13,29 Ostensibly, some of these factors can disproportionately affect patient care pipelines in subspecialized practices, particularly in instances where there is no widespread diagnostic expertise outside of the affected subspecialty group and/or in subspecialty groups with high numbers of older individuals. These factors together with potential regional and institutional demands in the face of COVID-19 (e.g. deployment of laboratory staff and physicians as part of institutional surge planning initiatives) could pose unexpected system-wide complications in the delivery of anatomic pathology services and overall patient care. 17,30,31 The University of Washington neuropathology division (UWNPD) covers three main medical centers within the University of Washington (UW) system across Seattle Washington and serves as a referral center for western Washington and a number of outside clients throughout the 5 state WWAMI region (Washington, Wyoming, Alaska, Montana and Idaho). Faced with this unprecedented situation and in order to mitigate potential patient care interruptions, UWNPD proposed a plan to departmental leadership for the rapid validation and deployment of telepathology services via the Leica Aperio slide imaging system. This plan builds upon existing workflows for digital pathology that have been employed by UWNPD for over a decade for the purpose of providing intraoperative consultation support throughout our Seattle-based hospitals.
A whole slide image (WSI) is a digitized high-resolution replica of a histopathology glass slide created on a whole slide scanner (WSS), which can be manipulated through software to mimic traditional microscope review and issue diagnoses. 32 Several published studies have reported low major and minor discrepancy rates demonstrating that diagnoses rendered by WSI are not inferior to primary diagnoses via traditional histopathology. [33][34][35][36][37][38][39][40][41][42][43][44][45] Per the College of American Pathologists (CAP; GEN.52920), each laboratory must validate WSS/WSI platforms for each clinical diagnostic usage by performing its own validation study. 46 Due to the COVID-19 pandemic, CAP initiated advocacy efforts geared towards a temporary waiver of current regulations limiting remote review of pathology slides in sites physically outside of the Clinical Laboratory Improvement Amendments (CLIA) accredited laboratory as set by the Centers for Medicare & Medicaid Services (CMS) and the Department of Health and Human Services (HHS). 47 On March 26, CMS, recognizing the need for expanded precautions to halt transmission in the workplace together with the wide-ranging operational challenges posed by the COVID-19 pandemic, issued a memorandum describing a policy of temporary enforcement relaxation of these regulations, thus allowing for remote "sign out" by pathologists via telepathology. 48 Lastly, on April 24, the Food and Drug Administration (FDA) announced to Leica Biosystems their intention to exercise enforcement discretion in regards to the use of Aperio WebViewer, a component of Aperio eSlide Manager, to examine slides acquired with an Aperio ScanScope AT2 and AT2 DX for remote pathology diagnosis.
Herein we describe a minimal resource utilization method for rapid validation and deployment of WSI technology for issuing neuropathologic diagnoses undertaken as part of our regional COVID-19 response during the upward sloping phase of regional and hospital resource utilization. This study can serve as a model for rapid and efficient telepathology subspecialty validation for laboratories that are preparing to face or are currently facing COVID-19 as part of an initial wave or potentially subsequent episodic waves, a combined flu and COVID-19 season, other mass casualty events, or workplace transmission events. 14,31,49

Study roles and training
The The DPM was also in charge of the training of the VPS, SPG and STG end-users as well as the technical training of the APT. Training sessions were performed prior to slide digitalization and diagnostic examination of cases as required for each of the participant's role in the study. Dated documentation of training for each specific role and each specific user is available on our laboratory's document control system (MediaLab, Inc., Georgia, USA) in accordance with CAP regulations (GEN.52900). In short, APT scanner training included hardware and software overview, slide preparation and loading into the scanner, review of pre-scan snapshots, tissue and focus point selection, quality control (QC) check of scanned images, and image assignment in Aperio eSlide Manager digital slide information system (Leica Biosystems, Inc., Illinois, USA). Prior to examination of cases, the VPS, SPG and 6 STG underwent training via one teleconference session covering the WSI system software. This session included topics on how to access the digitized slides from the UW Aperio eSlide Manager digital slide information system, operation of the Aperio viewing software programs (Aperio eSlide Manager and Aperio ImageScope), and WSI image quality control. Because the aim of this validation was to test for remote diagnostic usage of the WSI system, the study pathologists were allowed to use their own home devices when examining digital slides. The SPG and STG were provided with an overview of minimum computer monitor requirements and asked that they evaluate their home device to ensure they were confident in using the display when examining digital slides for diagnosis. Aperio eSlide manager and Aperio ImageScope fulfill the safeguarding of patient confidentiality under CAP regulations (GEN.52842). Following the completion of the validation study, the DPM and VPS held a final meeting/training digital session to address any outstanding topics related to image manipulation, software troubleshooting of any other potential issues identified during the validation study.

Validation study design
Case Curation-The VPS selected potential validation cases out of his personal collection or from sources available within the UW neuropathology division didactic repository consisting of over 500 cases (e.g. glass slide study sets and/or cases that had been set aside for various didactic purposes). Prior to inclusion in the study, all glass slides from each case were reviewed by the VPS in order to 1) verify that the original neuropathologic diagnosis and ancillary study interpretation was correct, 2) confirm the presence of the desired diagnostic features, 3), assess the preservation, sectioning and staining quality of all the slides, and 4) assess the quality of any included positive controls.
In order to best emulate the clinical environment of our intended use (providing diagnoses in an academic neuropathology practice) a total of 30 cases covering a wide spectrum of neuropathologic disease and slide preparations were selected [ Table 1]. These 30 cases were divided into two groups, then randomized within each group, deidentified and given a unique study case ID number. The Neurosurgical (Nsx) set was composed of 20 cases including neoplastic and non-neoplastic processes of the nervous system, eye and orbit, and brain autopsy pathology whereas the Neuromuscular (NM) set was composed of 10 cases including metabolic/mitochondrial and dystrophic processes as well as immune mediated myopathies/neuropathies. All cases contained at least one hematoxylin and eosin (H&E) stained slide with some cases containing variable numbers of ancillary studies (e.g. special stains (SS), immunohistochemical (IHC) stains (DAB chromogen), methylene blue-stained sections, and teased nerve fiber studies). The type and number of stains per case initially provided to the diagnosticians was at the discretion of the VPS, who selected these with the intent of providing diagnosticians with sufficient information to arrive and work through a differential diagnosis. Once case selection was finalized the DPM and VPS created a key to the randomized study case ID numbers containing case identifiers, neuropathologic diagnoses, and ancillary study interpretations that were extracted from the pathology report and was not shared with any other study members.
Study Materials Preparation-Each case included in our study was linked to its original neuropathology report in our laboratory information system (LIS; PowerPath, Sunquest, AZ, USA). For each case, the VPS extracted the patient's clinical information removing all patient identifiers and, when applicable, selected and provided images (e.g. radiologic studies, autopsy brain cutting photographs, electron microscopy photographs). It is important to note that as part of our divisional workflow, in particular for the workup of neoplastic neurosurgical cases, representative pre-surgical magnetic resonance imaging (MRI) images are routinely uploaded into the LIS.
As such providing these same radiological, ultrastructural or brain cutting images to the diagnosticians does not deviate from the information that was available at the time of the original neuropathological diagnosis. Also similar to traditional clinical practice, if a case had an intraoperative consultation this information was also made available to the diagnosticians. The case specific clinical information and images were assembled into one document per cohort and provided to the diagnosticians ( Figure 1A, B). The VPS also created an answer sheet for each set that included a section to rate each case's WSI quality (Figure 2A, B). The Nsx answer sheet included sections for overall disease classification and final diagnosis as well as, when applicable, tumor grading and interpretation of ancillary studies. Similarly, the NM sheet included sections for overall disease classification, final diagnosis and ancillary study interpretation. In addition, since NM cases are typically more complex, an additional column designated "any additional diagnostic comments" was added allowing the diagnosticians to voluntarily share any important clinicopathologic details or diagnostic comments.
Slide Digitization-Slides from each of the previously randomized and deidentified cases were then relabeled first using opaque labels, then with a second label displaying the assigned study case ID number and slide preparation (e.g. H&E, GT, CD20, etc.). The APT cleaned the blinded glass slides so they were free of marker dotting, dust, oils, or other contaminants that may affect WSI quality. Following this, the APT members performed a whole slide digitization of the slides at 40X magnification using the Leica Aperio ScanScope AT2 system (Leica Biosystems, Inc., Illinois, USA). During the scanning steps, an APT member reviewed the pre-scan snapshots of each slide to ensure all tissue on the slide was adequately scanned (including on slide positive controls for IHC and SS when applicable). After slide scanning and image acquisition, the technologist reviewed the digitized whole slide images for quality issues (focus, calibration errors, etc.) to ensure image quality control.
The technologist marked the time at which the slides are obtained and scanning is completed so as to measure turnaround time for scanning and ultimately interpretations.

Validation study execution
Each member of the SPG and STG independently reviewed the medical histories, clinical images, and digitalized slides using the Aperio eSlide Manager and/or Aperio ImageScope and recorded their interpretations on the validation study answer sheet. In this validation study, the slide labels were deidentified; as such, each end-user must use the study case ID on the image capture of the slide label as an identifier in order to confirm positive identification of the correct case. Quality of WSI was recorded for each case as either yes or no (in terms of adequacy to render primary diagnosis). The interpretations of all SPG and STG participants were analyzed by the VPS. Similar to traditional clinical practice SPG and STG members had the ability to request additional slide preparations that would aid them in making a diagnosis; however, dissimilar from a traditional clinical practice, diagnosticians were not allowed to seek slide review by other members of the team.

Interpretation of Results
The VPS evaluated each set of answers for diagnostic concordance between the WSI interpretations and the verified baseline diagnosis rendered via traditional histopathology. Areas of evaluation included general disease classification and final diagnosis as well as grade (when applicable). Evaluation of ancillary study interpretation was undertaken with a high level of scrutiny, emphasizing on not only the stated result but also the identification of diagnostic pertinent positive or pertinent negatives features for each case. A concordance threshold of ≥95% between the original and WSI diagnoses is required for positive validation. The final diagnosis results were assigned a quality concern score (QCS); concordant results are those with QCS of 0-2, and discordant results are those with QCS of 3-5. A QCS score of 0 indicates no quality concerns, 1 indicates a variation in practice that would is not associated with a clinically significant risk to the patient, and a 2 is a variation in practice associated with a clinically significant increased risk to patient. QCS scored of 3-5 would indicate errors that would result in additional care, were potentially life threatening or may cause other severe outcomes including death. Lastly, the DPM compiled the results, which are then reviewed and approved by the Medical Director.

Overview
Our rapid validation study is composed of 30 total cases (20 combined neurosurgical, autopsy neuropathology and ophthalmic and 10 neuromuscular cases) encompassing the wide range of pathologic processes and tissue/slide preparations that would be expected in an academic neuropathology practice. There was 100% diagnostic concordance without any major or minor diagnostic discrepancies for any of the SPG or STG members; as such discrepancy rates by diagnostician were not calculated. These results demonstrate that the WSI system was able to adequately capture the pathologic features of each slide/case and that each diagnostician was able to utilize these images together with the available clinical information to render correct diagnoses and ancillary study interpretations, as they would do in their traditional clinical practice. The validation study results are summarized in table 1.

Study Pathologist Group (SPG)
There were no major or minor diagnostic discrepancies in either the Nsx or NM sets (100% diagnostic concordance; QCS 0) and all ancillary studies were interpreted and described correctly. For case NM-10 one diagnostician commented that while they had identified vascular changes in the muscle and nerve that were concerning for vasculitis, in a clinical setting they would request additional studies in order to arrive at that diagnosis. Based on their request they were provided additional WSI generated from archival material that had been available to the original diagnostician; two additional formalin fixed paraffin embedded (FFPE) H&E stains (muscle and nerve) and IHC for the frozen muscle (CD3) and FFPE nerve (CD68). The SPG member reviewed these new sections and arrived at the correct diagnosis in both the muscle and nerve specimens. All study participants were given the opportunity to review these additional slides and none provided diagnostic modifications. Lastly, in the NM set answer sheets, none of the additional text provided in the "any additional diagnostic comments" section showed any major or minor diagnostic discrepancies.

Study Trainee Group (STG)
There were no major or minor diagnostic discrepancies in either the Nsx or NM sets (100% diagnostic concordance; QCS 0) and all ancillary studies were interpreted and described correctly. For case NM-03 one trainee correctly classified it as an "inflammatory myopathy", offered a short differential diagnosis including polymyositis and overlap myositis and suggested "…correlation with autoimmune testing may be helpful". In addition, the trainee correctly interpreted all ancillary studies (acid and alkaline phosphatases and HLA-I, CD3 and CD20 IHC). All other study participants also correctly diagnosed and characterized this case as an immune myopathy with perimysial pathology (IMPP). While IMPP is not a universally accepted term, it describes a phenotype that can be seen in some anti-synthetase syndrome patients (e.g. with anti-Jo1 antibodies) who can also be at risk for developing idiopathic pulmonary fibrosis. 50 As such and since this specific terminology is not uniformly accepted, a diagnosis of inflammatory myopathy and request for correlation with serological testing (e.g. myositis panel) is not considered a discrepancy nor does it pose any significant risk to the patient as these serological panels are routinely performed in patients with myositis. Lastly, it should be noted that STG members were given the opportunity to review the additional slides that were requested and provided for case NM-10 as part of their initial review.

Image Quality Assessment
Study participants did not defer any cases due to issues with image quality. As part of the validation study the SPG and STG members were also asked to assess WSI quality and render either a "yes" or "no" opinion as to whether the image quality was acceptable for diagnostic purposes; all members of the SPG and STG answered "yes" for all 93 slides on both the Nsx and NM sets. In general, all diagnosticians were impressed with the image quality of H&E and ancillary preparations. Additional image quality comments were requested and provided even if the diagnostician felt the WSI were of diagnostic quality. It must be noted even when diagnosticians provided additional comments related to image quality, these did not affect their ability to arrive at the correct pathological classification, diagnosis, and grade, nor did it affect their ability to provide the correct interpretation for each ancillary tissue preparation.
Questions regarding minimal adjustments to image brightness and/or contrast were the most commonly provided comments. For case Nsx-04 one STG member mentioned assessing the staining intensity for nuclear p53 immunoreactivity was perhaps "a bit harder to interpret for me than the others". For case Nsx-06 one SPG member mentioned that VVG was dark and somewhat difficult to interpret, similarly one STG member mentioned that the mGT stains on cases NM-01 and NM-04 were somewhat dark. In addition, one SPG member mentioned that for case Nsx-04, due to the newness of using this platform for mitotic counting, a Ki-67 (not available for this historic case but currently customarily ordered in all infiltrating gliomas in our clinical practice) would have been useful to corroborate their impression of low mitotic activity and confirm grade assignment. As mentioned on the method's "study roles and training" section, following the completion of the validation study the DPM and VPS held a wrap up digital session addressing topics related to software or troubleshooting of brightness, contrast, and color balance using the aforementioned cases as examples, which answered the specific diagnostician's questions.

DISCUSSION
Even with effectively deployed and adhered to community mitigation strategies, a multitude of regions and medical systems across the globe are facing unprecedented stress to their social infrastructure and healthcare systems during the COVID-19 pandemic. 14,15,49,[51][52][53] Services provided by anatomic pathology laboratories are a crucial component of the patient care pipeline even when not in direct contact with COVID-19 patients. 54 This is especially true for other urgent non-COVID-19 patients with serious medical conditions for whom pathologic diagnoses and integration of histopathological and molecular data are crucial for timely initiation of appropriate therapy regimens. 54 The combination of the nature of the work and interactions that occur in these laboratories, together with practice subspecialization, workforce demographics, and inherent characteristics of the SARS-CoV-2 virus could potentially lead to unexpected operational challenges (e.g. interruption of laboratory operations and/or diagnostic subspecialty services). 10,13,[27][28][29] This rapidly evolving situation has allowed our medical systems a variety of opportunities to meet these demands in flexible and innovative ways, while at the same time remaining compliant with regulatory guidelines, in the face of the extraordinary challenges posed by this pandemic. 14,15,49,52 In preparation for any of the aforementioned potential stressors, we designed and conducted, in roughly three weeks, a rapid validation study to assess the possibility of providing remote diagnostic neuropathology services via telepathology for the UW Medicine system. Our intent was to conduct this study with the least amount of additional resources as it was piloted in mid March to early April during the upward sloping of the epidemiologic and hospital resource utilization curves during the initial COVID-19 outbreak in western Washington. This validation study included the review of 93 slide preparations (e.g. H&E, special stains, immunohistochemistry, etc.) across 30 cases consisting of a mixture of neurosurgical, ophthalmic, brain autopsy, and neuromuscular pathology entities by a group of Neuropathology attendings and trainees. None of the diagnosticians had major or minor diagnostic discrepancies in either cohort (Nsx and NM); as such, achieving our threshold of at least 95% diagnostic concordance and successfully validating this platform within the Neuropathology subspecialty service during the COVID-19 pandemic as well as the period of relaxed CMS and FDA enforcement of policies regulating remote review of pathology slides.
Our rapid validation study has some limitations. First, this project builds upon the extensive collective telepathology experience of the UWNPD members (anywhere from 2.5-11 years per member), who when on service perform remote intraoperative diagnostic consultations using a similar platform (most recently Aperio LV1). As such, this level of user experience with telepathology may not be necessarily available in all other centers attempting to rapidly implement this system with limited resource utilization. An additional point is that this validation method did not follow the recommendation to establish pathologist intraobserver concordance between traditional review of glass slides and digital examination of WSI, but rather demonstrated pathologists' digital competency with the WSI system and interobserver concordance. 46 This compromise allowed for case selection from readily available on-site slide libraries, which both limited the number of staff interactions to support the institution's social distancing guidelines and addressed the needs of a rapid validation timeline. A more traditional validation study would require additional case selection and slide pull steps utilizing slide room staff, glass slide examination followed by a minimum 2-week washout period, and then digital slide examination. Another limitation is that the total number of cases evaluated in this study might be considered a constraint when compared to other larger studies; however, it should be noted that others groups have conducted validation studies which included a similar number of cases and slides (12-52 cases and 24-391 slides). [37][38][39][40]55 For our group an element of this choice was due to our desire to utilize the least amount of resources together with the need for rapid validation, during the time of telepathology related waivers, as a component of our COVID-19 planning response.
Consequently, we turned our attention to building very inclusive case cohorts with a broad range of neuropathologic entities and slide/tissue preparations in order to ensure a wide and robust representation of pathologic processes and preparations proper of an academic neuropathology practice. While some of the study set cases, individually, may not be reflective of all the complexities found in routine clinical practice we believe that for the reasons listed above the case selection acts to expand the spectrum of entities and preparations being validated within 30 neuropathology cases compared to what could be achieved with a set of consecutive cases.
In addition, our validation study used 40X magnification slide scans, and thus all digital slides used for clinical diagnostic purposes under this validation will need to be scanned at this resolution. We note this resolution may not be necessary for all subspecialty services and as such the impact of slide scanning resolution on the daily throughput of the scanner, image storage, and bandwidth requirements should be considered when laboratories are designing their clinical workflows guided by our validation study. It should also be noted that while this study included immunohistochemical slide preparations using the DAB chromogen, preparations using different chromogens or prognostic biomarkers (HER2, ER, PR) should and will be validated separately to ensure the digital pathology system is able to capture the expected range of colors and intensities for the intended diagnostic application. Notwithstanding, we note the need for additional validations in order to study supplementary organ systems and diagnostic applications/specimen preparations. 56 In order to address these limitations the UW Digital Pathology section is currently working on a follow-up validation study, with our colleagues in the genitourinary pathology subspecialty, which is designed to meet all recommendations on a more relaxed timeline.
Once deployed, diagnostic WSI services would need to be included in the laboratories quality management plan as mandated by CAP (GEN.52860). A policy for prospective review of cases in which telepathology was used to arrive at a primary diagnosis is still being formalized at a departmental and subspecialty level. Currently in UWNPD, over 80-90% of neoplastic cases are reviewed at our weekly case review conference; a subset of which are then reviewed again during multidisciplinary conference preparation. Occasionally, interesting and/or challenging non-neoplastic cases may also be reviewed in this weekly forum, in our monthly ophthalmic or neuromuscular case review conferences, or in preparation of other working conferences within the UW system.
For UWNPD, these safeguards would allow an adequate degree of oversight. Part of our future policy would