{'fraction_non_alphanumeric': 0.035210237659963436, 'fraction_numerical': 0.025850091407678244, 'mean_word_length': 5.153205849268842, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Initial determination of COVID-19 seroprevalence among outpatients and healthcare workers in Minnesota using a novel SARS-CoV-2 total antibody ELISA

Objectives To avoid the significant risks posed by the use of COVID-19 serology tests with supply chain constraints or poor performance characteristics, we developed an in-house SARS-CoV-2 total antibody test. Our test was compared with three commercial methods, and was used to determine COVID-19 seroprevalence among healthcare workers and outpatients in Minnesota. Methods Seventy-nine plasma and serum samples from 50 patients 4–69 days after symptom onset who tested positive by a SARS-CoV-2 PCR method using a nasopharyngeal (NP) swab were used to evaluate our test’s clinical performance. Seropositive samples were analyzed for IgG titers in a follow-up assay. Thirty plasma and serum from 12 patients who tested negative by a SARS-CoV-2 PCR method using a nasopharyngeal (NP) swab and 210 negative pre-pandemic serum samples were also analyzed. Among samples from patients > 14 days after symptom onset, the assay had 100% clinical sensitivity and 100% clinical specificity, 100% positive predictive value and 100% negative predictive value. Analytical specificity was 99.8%, indicating minimal cross-reactivity. A screening study was conducted to ascertain COVID-19 seroprevalence among healthcare workers and outpatients in Minnesota. Results Analysis of serum collected between April 13 and May 21, 2020 indicated a COVID-19 seroprevalence of 2.96% among 1,282 healthcare workers and 4.46% among 2,379 outpatients. Conclusions Our in-house SARS-CoV-2 total antibody test can be used to conduct reliable epidemiological studies to inform public health decisions during the COVID-19 pandemic.


Introduction
SARS-CoV-2 is the viral causative agent of COVID-19, which is currently a global pandemic. [1] COVID-19 serology or antibody tests are a key tool for elucidating an individual's or community's exposure history to SARS-CoV-2, through detection of an immune response from a current or past infection with SARS-CoV-2. Identifying this population is important because SARS-CoV-2 polymerase chain reaction (PCR) tests only detect the presence of viral nucleic acid in individuals with active infections. It is currently estimated that as many as 25% of those who are infected are asymptomatic; therefore antibody testing can provide comprehensive data on true rates of exposure to SARS-CoV-2. [2,3] Potential uses of serology assays include community screening, contact tracing, epidemiological studies, and screening convalescent plasma collected from individuals who have recovered from COVID-19. [4] In response to the public health emergency related to COVID-19, the Food and Drug Administration (FDA) invoked the Emergency Use Authorization (EUA) pathway to accelerate the availability of COVID-19 tests. Although FDA EUA is required for clinical applications of SARS-CoV-2 molecular tests, until May 4, 2020, manufacturers of serologic assays were submitting for EUA only on a voluntary basis. [5] As a result, several serology tests were marketed that were not approved under the EUA, did not provide accurate results, had high false positive rates, and were not independently validated. [6] To avoid the significant risks posed by the use of commercially available tests during the early stages of the pandemicsome of which had poor performance characteristics and did not have EUA designation and potential associated supply chain disruptions, we pursued a multidepartmental effort within the University of Minnesota to develop a SARS-CoV-2 total antibody test using an enzyme-linked immunoassay (ELISA) format. First, we examined what is known about SARS-CoV-2 and other coronaviruses. Coronaviruses share structural similarities and are composed of 16 non-structural and four structural proteins, which are the spike (S), envelope (E), membrane (M), and nucleocapsid (N) proteins. [7] Common to all coronaviruses is a receptor binding domain (RBD) within the spike protein; prior studies have found that the RBD is a common target for neutralizing antibodies with the closely related SARS and MERS viruses. [8] For SARS-CoV-2, the S protein and RBD have unique features relative to other coronaviruses for enhanced cell entry, including an RBD within the S protein that can undergo furin pre-activation for cell entry and the RBD's high angiotensin converting enzyme 2 (ACE2) binding affinity. [9] Given the key role of the RBD in viral pathogenesis and known antigenicity among other closely related coronaviruses, we utilized a recombinant SARS-CoV-2 RBD protein fragment to produce antigen for the development of a manual ELISA within the University of Minnesota's Center for Immunology. [10,11] The ELISA was then transferred to the University of Minnesota's Advanced Research and Diagnostic Laboratory where clinical validation was performed to enable the assay to be used for patient testing with a current capacity of 2,000 tests per day.
As part of the assay validation, we conducted a method comparison study with three commercial SARS-CoV-2 antibody assays that utilize various SARS-CoV-2 antigens (N, S protein), detect different antibody classes (IgG, total antibody), and have different assay formats (ELISA and chemiluminescent microparticle immunoassay). Although these commercial assays have been used in large-scale seroprevalence studies in metropolitan cities worldwide, there is a critical lack of data demonstrating their comparative diagnostic performance. The use of seroprevalence data in guiding public policy decisions underscores the importance of comprehensively delineating the relative clinical sensitivity and specificity of these SARS-CoV-2 serology tests. Our method comparison study was designed to provide this critical data.
With our robustly validated laboratory-developed serology test, we set forth to establish an initial determination of COVID-19 seroprevalence among healthcare workers and outpatients in Minnesota. Here, we present the results from the first 38 days that this test was available for the afore-mentioned populations.

Study approval and specimen acquisition
The University of Minnesota Institutional Review Board (IRB) determined that this study was not considered human research. Clinical validation of the ELISA assay entailed use of remnant serum, lithium heparin plasma and EDTA plasma specimens with associated clinical identification obtained from M Health Fairview acute care clinical laboratories serving hospitalized COVID-19 patients. COVID-19 positive (n = 79 samples from 50 patients: 38 Lithium Heparin plasma, 32 serum, 9 EDTA plasma) or negative (n = 30 samples from 12 patients: 28 Lithium Heparin plasma, 2 serum) patients were categorized based on SARS-CoV-2 PCR test results (Fig. 1). SARS-CoV-2 testing was conducted using four methods. Three of these methods have FDA EUA designation (CDC 2019-nCoV RT-PCR Diagnostic Panel, Initial EUA issued 02/02/ 2020; Roche Molecular Systems cobas SARS-CoV-2, EUA issued 03/12/ 2020; DiaSorin Molecular Simplexa COVID-19 Direct, EUA issued 03/ 19/2020), and one method is under evaluation for EUA approval (EUA 200,153 University of Minnesota Genomics Center (UMGC) COVID-19 qRT-qPCR assay). Blood samples from patients meeting these criteria were identified using the laboratory information system (Sunquest, Sunquest Information Systems, Tucson, AZ). Serology testing is prioritized by the M Health Fairview healthcare system to ensure patients are tested in a manner that most significantly impacts the system's ability to understand disease spread and evaluate higher-risk individuals. For our seroprevalence study, we analyzed 3,661 serum samples received during the first 38 days that our assay was available as an orderable clinical test. These samples were obtained from healthcare workers with confirmed and non-confirmed COVID-19 exposures ≥ 14 days prior, and asymptomatic outpatients with potential COVID-19 exposures or history of prior symptoms consistent with COVID-19 ≥ 14 days prior.

Preparation of antigen for ELISA
The RBD fragment from the previously published literature [10,11] was expressed in insect cells. To improve the antigenicity of the RBD fragment, we re-expressed the same RBD fragment in HEK293T cells. To this end, HEK293T cells stably expressing RBD (containing a human Fc tag) were made according to the E and F section of the pLKO.1 Protocol from Addgene (http://www.addgene.org/protocols/plko/). The Fctagged RBD was then purified as previously described. [10,11]

Data analysis
Patient demographic data were extracted using Sunquest and clinical data including patient-reported COVID-19 symptoms and date of symptom onset were extracted from electronic health records (EPIC, EPIC Systems Corporation, Verona, WI). COVID-19 symptoms included fever, cough, fatigue, shortness of breath and myalgia. [16] Serum samples from 210 individuals collected between September 2018 and March 2019 (prior to the emergence of the COVID-19 pandemic) and stored at − 80 • C were used as presumed COVID-19 true negative specimens, and an additional 310 specimens that were positive for other infectious agents or autoimmune antibodies were used to determine the analytical specificity/cross-reactivity. Specimens from patients who tested negative or positive by a SARS-CoV-2 PCR method using a nasopharyngeal (NP) source were considered as true negatives or true positives, respectively. The diagnostic performance of the SARS-CoV-2 total antibody ELISA was determined

Analytical assay performance
Our in-house SARS-CoV-2 total antibody ELISA is a qualitative screen for total antibodies to the spike RBD antigen, with the semi-quantitative measurement of IgG antibodies by endpoint titer (Supplemental Fig. 2). Serum, lithium heparin and EDTA plasma are acceptable specimen types (Supplemental Methods). The limit of quantification (functional sensitivity) is 10 ng/mL. The screening test was designed to recognize IgG, IgM and IgA antibodies as the combined detection of these immunoglobulins has a higher sensitivity than the detection of each antibody alone in the setting of COVID-19. [19][20][21] Although it has been shown that heat inactivation of serum interferes with the detection of antibodies to SARS-CoV-2 potentially causing false-negative results, we did not observe a significant difference in the antibody indices of 10 samples that were analyzed with and without heat inactivation (56 • C for 1 h; p > 0.05; t-test). [22] Analytical specificity/cross-reactivity was evaluated by testing 520 blood specimens from patients with antibodies to 17 other common viral infections and autoantibodies, which could potentially cause false positive results: Herpes Simplex Virus IgG (n = 10), Mitochondrial IgG (n = 10), Cytomegalovirus IgG (n = 12), Varicella Zoster IgG (n = 15), Epstein Barr Virus IgG (n = 5), Rubella IgG (n = 13), Mumps IgG (n = 3), Rubeola IgG (n = 3), Human immunodeficiency virus 1/2 antibody (n = 10), Hepatitis B core antibody (n = 5), Hepatitis B surface antibody (n = 13), Hepatitis C antibody (n = 2), Haemophilus influenza type B IgG (n = 23), Influenza A IgG (n = 88), and Influenza B IgG (n = 77), and a prepandemic cohort used to assess for cross-reactivity to common cold coronavirus antibodies (n = 210). Samples were also obtained from 21 patients who tested positive for Hepatitis B virus DNA. Samples with direct evidence of antibodies to the common cold coronaviruses were not available for testing. However, the seroprevalence for the common cold coronaviruses is high (60-90%); therefore, cross-reactivity was indirectly ruled out through testing 210 pre-pandemic samples. [23][24][25] A total of 519 out of 520 samples (99.8%; 95% CI: 98.92 -99.97%) showed no cross-reactivity (Table 1) indicating high analytical specificity.

Clinical assay performance
The diagnostic performance of the SARS-CoV-2 total antibody ELISA was assessed using plasma and serum samples from patients with clinical presentations suggestive of COVID-19 infection who had positive or negative SARS-CoV-2 PCR tests, and serum from presumed true negative samples that were collected between September 2018 and March 2019 (prior to the emergence of the COVID-19 pandemic). A total of 79 blood samples (38 Lithium Heparin plasma, 32 serum, 9 EDTA) were analyzed from 50 patients 4-69 days after symptom onset who tested positive by a SARS-CoV-2 PCR method using an NP swab as determined by review of the electronic medical record. Blood samples with negative or equivocal antibody indices ≤ 4.0 from patients with PCR positive NP specimens were classified as "PCR positive Low", whereas blood samples with positive antibody indices > 4.0 from patients with PCR positive NP specimens were classified as "PCR positive High". A total of 237 plasma and serum samples were analyzed from 12 patients with SARS-CoV-2 PCR negative NP specimens (28 Lithium Heparin plasma, 2 serum) and 207 presumed true negative pre-pandemic samples (all sera) from 207 patients (Fig. 1). The SARS-CoV-2 total antibody ELISA has an overall 86.1% clinical sensitivity (95% CI: 76.4% -92.8%), and 100% clinical specificity (95% CI: 98.5%-100.0%). The positive and negative predictive values based on a disease prevalence of 1.5% or 5% are indicated in Table 2. Among patients with PCR positive NP specimens, stratification by days after symptom onset yields a clinical sensitivity of 71.8% (95% CI: 55.1%-85.0%), 100% clinical specificity (95% CI: 98.5% -100.0%), for patients between 4 and 14 days after symptom onset. The assay has the most robust performance characteristics when used with patients > 14 days after symptom onset: 100% clinical sensitivity (95% CI: 91.2%-100.0%) and 100% clinical specificity (95% CI: 98.5% -100.0%).
The clinical sensitivity of our assay increases with the number of days post-COVID-19 symptom onset (Supplemental Fig. 3). Both patients whose SARS-CoV-2 antibody temporal profiles are depicted in Supplemental Fig. 3 had positive diagnostic PCR tests six days after symptom onset. Patient 1 had a 31-day hospital course after ICU admission prior to being discharged, whereas patient 2 had a 16-day hospital course from ICU admission to discharge.
Whereas SARS-CoV-2 viral RNA is detectable even prior to symptom onset and reaches its peak at day 5 after symptoms, antibody responses begin near day 7 and most patients exhibit rising IgG and IgM antibody titers 10 days after symptom onset. [20,26] Because of what is known todate regarding the viral kinetics and antibody responses in patients with COVID-19, [27] our SARS-CoV-2 total antibody ELISA is not recommended for use in patients within 10 days of symptom onset as they may produce insufficient levels of detectable antibodies.
Equivocal results indicate that antibodies were detected at a level close to the threshold of the limit of detection for the assay. Such results could represent an early stage of SARS-CoV-2 infection, detection of decreasing antibody levels, cross-reactivity with viral antibodies not included in the method validation studies, or a weak antibody response among immunosuppressed patients or patients with an underlying immune disorder. Repeat testing of patients with equivocal results with additional blood samples at a later date is recommended if clinically indicated.

Comparison of laboratory-developed SARS-CoV-2 total antibody ELISA with commercial serology assays
We compared our laboratory-developed total antibody ELISA to three commercially available serology assays using plasma and serum samples from a total of 35 SARS-CoV-2 patients 4-35 days after symptom onset who tested positive by a SARS-CoV-2 PCR method using an NP swab and 68 presumed true negative pre-pandemic samples. A total of 20, 43, or 60 samples were used for each comparison. The positive samples included seroconversion samples that were obtained from the Table 1 Cross-reactivity of SARS-CoV-2 total antibody ELISA.   Table 3. Our laboratory-developed ELISA assay demonstrated excellent agreement with the compared commercial assays ( Table 4). Given that the range of antibody indices corresponding with the presence or absence of antibodies differs for each assay, the results were evaluated based on the categorical variables of "positive" and "negative"; equivocal results were omitted from the analysis. The results showed that our laboratory-developed total antibody ELISA exhibited 100% overall agreement (20/20; Cohen's kappa = 0.99) with the Epitope Diagnostics SARS-CoV-2 nucleocapsid protein IgG assay, 94.8% (55/58; Cohen's kappa = 0.89) with the Abbott Architect SARS-CoV-2 nucleocapsid protein IgG method, and 92.1% (35/38; Cohen's kappa = 0.83) with the Euroimmun SARS-CoV-2 spike protein IgG assay. For the three discordant results between our laboratory-developed ELISA and the Euroimmun assay, the negative results from our assay are likely correct as these samples were from the pre-pandemic cohort. The one sample with a negative result from our laboratory-developed ELISA that had a positive result when using the Abbott assay was also from the pre-pandemic cohort. The two samples with positive results from our laboratorydeveloped ELISA that had negative results when using the Abbott assay were obtained from SARS-CoV-2 patients who tested positive by a PCR method using an NP swab. These discordant results could be explained by the difference in the epitopes and isotypes targeted by the different assays (the Abbott assay detects IgG whereas our laboratorydeveloped ELISA detects IgG, IgM, and IgA) and the sensitivity of the assays when testing patients who are within the early stages of disease progression.

Seroprevalence among healthcare workers and outpatients
The M Health Fairview healthcare system includes 10 hospitals and 60 clinics predominantly located in the Twin Cities, but inclusive of outlying cities throughout the state as well. As an initial step to a statewide COVID-19 testing initiative announced by the Governor of Minnesota to establish an estimate of the percentage of the population that has been exposed to COVID-19, we deployed our SARS-CoV-2 total antibody assay to test 1,282 healthcare workers and 2,379 outpatients within the M Health Fairview system during the first 38 days that our assay was available as an orderable clinical test ( Table 5).
The distribution of IgG titers in the outpatient population was not significantly different from the range of IgG titers among the healthcare workers (p = 0.60, Chi-square test) (Fig. 3). Additionally, there were no significant age-or sex-specific differences in seroprevalence in these two populations.

Discussion
Our study provides the first determination of COVID-19 seroprevalence among healthcare workers and outpatients in Minnesota. Although the 4.46% seroprevalence among this cohort of patients suggests that individuals with mild presentation of COVID-19 develop an antibody response, we do not have data regarding the percentage of these outpatients who required subsequent hospitalization due to COVID-19 related symptoms. As serology tests are deployed on a broader scale, the relationship between asymptomatic or mild forms of disease presentation and the development of an immune response will be more clearly defined.
Potential sources of variability among serologic assays include differences in acceptable specimen types (serum, plasma, whole blood), formats (ELISAs, chemiluminescent immunoassay, lateral flow immunoassay), detected antibody classes (IgA, IgM, IgG, total), and SARS-CoV-2 antigen(s) used to design the assay. [28] There is particular debate on whether assays should target the N or S protein -whereas the N protein might be expressed earlier in the viral lytic cycle and may have greater sensitivity earlier in the disease process than the S protein, the S protein could be more immunologically relevant due to its role as a target for neutralizing antibodies and vaccine development. [29] Despite these potential sources of variability, the results from our method comparison study demonstrated excellent concordance between our inhouse method and three commercial antibody methods. Importantly, our data suggest that the target SARS-CoV-2 antigen does not appear to significantly impact the accuracy or concordance of serology test results. This could have important implications for correlating various antibody test methods to protective neutralizing antibodies that primarily target the S protein RBD. In these cases, ELISAs using the SARS-CoV-2 N protein as the antigen could be a reasonable surrogate for measuring protective neutralizing antibodies, although additional confirmatory Table 3 Technical and performance specifications of assays used in method comparison study. studies are needed. In a recently published study, antibodies against the N protein showed 100% sensitivity and specificity in patients ≥ 15 days after symptom onset, whereas antibodies against the S protein were associated with 91% sensitivity and 100% specificity. [29] However, our data demonstrated that our assay, which detects total antibodies to the S protein RBD, exhibited equivalent diagnostic agreement with a commercial assay that detects IgG antibodies against the N protein (Epitope Diagnostics) even when samples were collected < 14 days of symptom onset.
The population of healthcare workers we tested for our seroprevalence study represents potential workplace COVID-19 exposures. The low seroprevalence (2.96%) in this population suggests that guidelines regarding the use of personal protective equipment (PPE) for these healthcare workers are effective at mitigating the spread of COVID-19. A seroprevalence study in Germany came to the same conclusion regarding the effectiveness of PPE, with a low seroprevalence rate of 1.6% among healthcare workers at a tertiary care hospital. [30] Acknowledging that a sampling of the outpatients within one healthcare system is not a random community sampling, our data suggest that healthcare workers could have a lower overall rate of infection compared to the general population in Minnesota. The results from a study investigating the rate of COVID-19 infection among healthcare workers in downstate New York demonstrated a similar trend. [31] Large-scale geographic COVID-19 seroprevalence surveys have been conducted in major metropolitan cities throughout the U.S. The seroprevalence in Santa Clara County, CA was 2.8% as of April 3-4, 2020. [32] Preliminary data suggest that COVID-19 infections are far more widespreadand the fatality rate much lowerin Los Angeles County than previously thought. The seroprevalence in this region was determined to be 4.1%. [33] Although these studies provide informative benchmarks for local disease prevalence, the commercial serology tests used in these studies have questionable performance characteristics, which impedes the reliability of the pursuant comparative studies. As additional seroprevalence data are obtained to support public health decision-making during the COVID-19 pandemic, it is imperative that these data are acquired using serologic test platforms with reliable performance characteristics.
Given the high sensitivity and specificity of our in-house developed test, the predictive value of our test will be robust even among populations with a low prevalence of exposure. Extrapolating the 4.46% seroprevalence in our sample of 2,379 outpatients to the population of Table 4 Concordance between the laboratory-developed total antibody ELISA and commercially available serology assays.   Fig. 3. Distribution of SARS-CoV-2 S1-RBD IgG titers among outpatients and healthcare workers. SARS-CoV-2 S1-RBD total antibody positive sera from outpatients (n = 106) and healthcare workers (n = 38) were analyzed by endpoint IgG titer using our laboratory-developed ELISA. The distributions of IgG titers in these populations were not significantly different (p = 0.60, Chisquare test).
Minnesota (5.64 million), [34] 251,500 individuals would be expected to have been exposed to COVID-19. However, this number is 10-times greater than the current number of laboratory-confirmed COVID-19 cases in the state as reported by the Minnesota Department of Health, [35] suggesting that COVID-19 is more widespread than reported. Although our study focused on the use of serology tests for seroprevalence determinations, the detection of SARS-CoV-2 antibodies also has an integral role in identifying protective antibodies with neutralization assays. Such assays provide quantitative information on the ability of patient antibodies to confer protective immunity based on the antibody-mediated inhibition of virus growth ex vivo. RBD-specific antibodies have previously been shown to exhibit neutralizing functions against SARS-CoV-2, supporting the likelihood for protective immunity. [36,37] However, future studies are needed to more rigorously demonstrate correlation between laboratory-developed and commercial antibody test results and neutralizing antibody titers.
Our study has limitations that should be considered when interpreting the results. First, remnant blood specimens were utilized for the assay validation studies. The majority of the specimens were analyzed 24-48 h following their initial collection after which time they were stored at 4 • C. However, the data from our stability studies indicate that these storage conditions did not cause significant changes in the levels of SARS-CoV-2 total antibody detected by our ELISA method. Second, the initial date of symptom onset was determined from subjective reports obtained from the patients at their time of hospital admission, as noted in the electronic medical record. However, this is a limitation that is common to several COVID-19 studies. Third, our method validation, which demonstrated 100% sensitivity and specificity after 14 days from symptom onset, was performed primarily with samples from PCR positive COVID-19 patients who required hospitalization. While our data on healthcare workers and outpatients demonstrates the ability of our method to detect antibody responses in milder cases, it is not clear if these sensitivity and specificity metrics apply for non-hospitalized PCR positive COVID-19 patients.
Preliminary studies suggest a correlation between serum antibody levels and clinical severity of disease. [38,39] Therefore, additional studies utilizing quantitative or semi-quantitative antibody methods like ours are needed to definitively establish the relationship between antibody levels and severity. Additional unanswered questions remain including the influence of co-morbidities on patients' COVID-19 immune responses, the impact of immunogenetic determinants on immune response, the time course of antibody production in the context of naturally acquired immunity, the kinetics of a protective antibody response stimulated by a safe and effective vaccine, and the identification and mechanism of action of neutralizing antibodies as prophylactic and therapeutic COVID-19 treatment.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.