{'fraction_non_alphanumeric': 0.03748849037576183, 'fraction_numerical': 0.03078002367694129, 'mean_word_length': 5.201196302338228, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Estimating COVID-19 Vaccination and Booster Effectiveness Using Electronic Health Records From an Academic Medical Center in Michigan

Introduction Observational studies of COVID-19 vaccines’ effectiveness can provide crucial information regarding the strength and durability of protection against SARS-CoV-2 infection and whether the protective response varies across different patient subpopulations and in the context of different SARS-CoV-2 variants. Methods We used a test-negative study design to assess vaccine effectiveness against SARS-CoV-2 infection and severe COVID-19 resulting in hospitalization, intensive care unit admission, or death using electronic health records data of 170,741 adults who had been tested for COVID-19 at the University of Michigan Medical Center between January 1 and December 31, 2021. We estimated vaccine effectiveness by comparing the odds of vaccination between cases and controls during each 2021 calendar quarter and stratified all outcomes by vaccine type, patient demographic and clinical characteristics, and booster status. Results Unvaccinated individuals had more than double the rate of infections (12.1% vs 4.7%) and >3 times the rate of severe COVID-19 outcomes (1.4% vs 0.4%) than vaccinated individuals. COVID-19 vaccines were 62.1% (95% CI=60.3, 63.8) effective against a new infection, with protection waning in the last 2 quarters of 2021. The vaccine effectiveness against severe disease overall was 73.7% (95% CI=69.6, 77.3) and remained high throughout 2021. Data from the last quarter of 2021 indicated that adding a booster dose augmented effectiveness against infection up to 87.3% (95% CI=85.0, 89.2) and against severe outcomes up to 94.0% (95% CI=89.5, 96.6). Pfizer-BioNTech and Moderna vaccines showed comparable performance when controlling for vaccination timing. Vaccine effectiveness was greater in more socioeconomically affluent areas and among healthcare workers; otherwise, we did not detect any significant modification of vaccine effectiveness by covariates, including gender, race, and SES. Conclusions COVID-19 vaccines were highly protective against infection and severe COVID-19 resulting in hospitalization, intensive care unit admission, or death. Administration of a booster dose significantly increased vaccine effectiveness against both outcomes. Ongoing surveillance is required to assess the durability of these findings.


INTRODUCTION
A total of 3 coronavirus disease 2019 (COVID-19) vaccines were developed, assessed for efficacy against symptomatic COVID-19 disease in placebo-controlled trials, and approved under emergency use authorization in the U.S. by February 2021: mRNA-1273 (Moderna), BNT162b2 (Pfizer-BioNTech), and Ad26.COV2.S (Johnson & Johnson-Janssen). 1−3 As of June 16, 2022, a total of 592 million doses had been administered, and 222 million people had been fully vaccinated in the primary series, meaning that they received at least 1 dose of Janssen or 2 doses of Pfizer-BioNTech or Moderna vaccine in the U.S. 4 Administration of booster doses has been shown to provide more protection than the primary series, 5,6 but only 47.2% in the U.S. had received a booster as of June 16, 2022. 7 Observational studies of vaccine effectiveness (VE) can assess real-world effectiveness, estimate duration of protection, identify protection against new variants that arise, and provide guidance on booster requirements. 8−16 Observational studies conducted early in 2021 through COVID-19-Associated Hospitalization Surveillance Network, a surveillance network in 13 states, including Michigan, found that 2 doses of Pfizer and Moderna provided 96% effectiveness in protecting against hospitalization, 17 and partial vaccination was 64% effective against hospitalization. 18 The Food and Drug Administration approved vaccination boosters, 19 but the protection that the booster doses provide over time and among different vaccines remains unclear. It is critical to assess whether VE and durability of protection vary by patient demographics, comorbidities, history of previous severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, SARS-CoV-2 exposure risk, and other variables that may impact vaccine performance. 20 Retrospective analyses using large health systems' electronic health record (EHR), and other similar databases, provide rich data for this type of research.
We used EHR data from the University of Michigan Health System (i.e., Michigan Medicine [MM]), a large, nationally ranked healthcare center in Michigan, to conduct a time-stratified retrospective cohort study to evaluate VE for the 3 major COVID-19 vaccines that were available in the U.S. Michigan is a state with both large urban population centers and rural communities and with extensive socioeconomic disparities across coun-ties¡particularly between Detroit, the most populous Black-majority city in the U.S., and its suburbs. In 32 (39%) of 83 counties in Michigan, vaccination primary series completion rates were below 50%. 21,22 The primary study goals were to determine the following: 1. VE against infection and severe disease across 2021; 2. VE stratified by the 2 most common vaccines¡Pfizer-BioNTech and Moderna¡and by sociodemographic and clinical characteristics that are associated with COVID-19 outcomes; and 3. VE of booster doses on the basis of the data from the last quarter of 2021.

Study Sample
In a test-negative design cohort, eligible individuals included adults (aged ≥18 years) who received primary or other health care at MM and had a reverse transcription-polymerase chain reaction (RT-PCR) test for SARS-CoV-2 infection performed or recorded at MM between January 1 and December 31, 2021. All individuals of this cohort were MM patients; however, some were also healthcare staff. The remaining 18,425 vaccinated individuals were excluded because they were partially vaccinated (i.e., had not completed the vaccination primary series [n = 18,321] or received a mixed sequence of vaccines outside of CDC guidelines 24 [n = 104]) or received unspecified or other vaccine brands (i.e., Astra Zeneca, Sinopharm, Sinovac, or Novavac) (n = 254) (Appendix Table 1 and Appendix Figure 1B, available online). Individuals were classified as being boosted if they received any additional vaccination at least 21 days after a completed vaccination series. We conducted the boosterrelated VE analysis only in Quarter (Q) 4 of 2021, covering the period before the CDC guideline for boosters was announced. 24,25 COVID-19 Outcomes. n this study, we focused on 2 COVID-19-associated outcomes: (1) SARS-CoV-2 infection and (2) severe disease or death. SARS-CoV-2 infection was defined as any documented positive result on an RT-PCR test (symptomatic or asymptomatic, nonsevere or severe disease), whereas severe disease or death was defined as either hospitalization or intensive care unit (ICU) admission between À7 and +30 days relative to the date of a documented positive SARS-CoV-2 RT-PCR test result or death between À7 and +60 days relative to the date of a positive RT-PCR test (Appendix Figure 1A, available online). Data on hospitalizations, ICU admissions, and deaths were obtained from MM's EHR databases as well as from the Michigan Death Registry.

RT-PCR Test
Demographics, Socioeconomic Status, and Other Covariates. We have access to all patients' structured deidentified EHR data that had been added by an MM healthcare provider. We performed additional analyses to examine whether VE varied by patient demographics (i.e., age, sex, race/ethnicity), weight status (i.e., BMI), clinical characteristics (e.g., comorbidities, history of SARS-CoV-2 infection, or history of cancer), occupational exposure (i.e., documented healthcare worker [HCW] or not), and SES (i.e., neighborhood disadvantage index [NDI] 26,27 and population density). Age, gender, and race/ethnicity were directly extracted from the EHR, whereas history of SARS-CoV-2 infection, history of cancer, immunosuppressed status, 25,28 BMI, the Elixhauser comorbidity score, 29,30 HCW status, NDI, and population density were derived on available variables. Definitions for derived variables are available in Appendix Text 2 (available online). We assumed that covariates included in our adjusted analyses were missing completely at random and performed complete case analyses for each adjustment. The sample sizes of the complete case analyses for various sets of covariates are listed in Appendix Table 4 (available online).
Ethical review and approval were waived for this study because of its qualification for a federal exemption as secondary research for which consent is not required. Determination for exemption was made by the University of Michigan Medical School IRB.

Statistical Analysis
We fit Firth's bias-corrected logistic regression for each COVID-19-related outcome Y COVID , considering several sets of covariates: We estimated the VE using the following formula 31 : where incidence rate ratio, the ratio of incidence between the vaccinated group and the unvaccinated group, can be estimated using the Michigan 12-month COVID-19 incidence rate and the estimated OR from the logistic regression mentioned earlier through the following formula 32 : We estimated an incidence rate of 0.088 on the basis of the cumulative case count through December 29, 2021 (1,710,325 cumulative cases; JHU COVID-19 Dashboard 33 and the resident population of Michigan: census 2020: 10,077,331). 22 We conducted a quarter-stratified analysis to examine the time-varying changes in VE over 2021: Q1, January 1−March 31; Q2, April 1−June 30; Q3, July 1−September 30; and Q4, October 1−December 31. For the estimation of the overall VE in 2021, we used the latest quarter in which a patient was tested (see above).
Finally, we estimated VE separately for individuals who received Pfizer-BioNTech only or Moderna only and provided estimates for the overall cohort as well as by quarter.
To assess the potentially different VE across different strata of risk factors (denoted as X) associated with COVID-19 outcomes, we further conducted interaction analysis by vaccine status using the following model: All analyses were performed in R statistical software, Version 4.1.2. 34 For each model, we reported VE along with the corresponding 95% Wald CIs and pvalues. For the interaction analysis, we obtained the vaccination OR for each stratum of X on the basis of b Vac þ b Int and reported the strata-specific subgroup VE along with the p-values, P int: , by testing the difference of subgroup effects through the null hypothesis H 0 : b int: ¼ 0. We performed a multiple testing correction on the basis of a Bonferroni-corrected threshold of p<0.00294 (0.05/17) to maintain the overall significance level at 0.05 across the total of 17 performed interaction tests.

Participant Characteristics
In our cohort of 170,487 adults who were tested for SARS-CoV-2 infection between January 1 and December 31, 2021, 74,060 (43.4%) were fully vaccinated, and 78,002 (45.8%) were unvaccinated. In addition, 7,187 (9.7%) individuals from the vaccinated group received at least 1 additional booster vaccine dose after completing the primary series. In the overall cohort, the average time between being fully vaccinated and a recorded test was 136 days. More than 66.5% of individuals were tested within 3 months of being fully vaccinated.
The characteristics of individuals by vaccination status are summarized in Table 1. Vaccinated individuals were slightly older (37.6% of the vaccinated group were aged ≥65 years compared with only 20% of the unvaccinated), were more likely to be established MM primary care patients (i.e., received a primary care visit at MM within the last 24 months; 45% vs 31.9%), and generally resided in less socioeconomically disadvantaged areas.
The vaccinated subgroup had a slightly higher proportion of individuals with multiple comorbidities (Elixhauser Index ≥5). Furthermore, nearly twice as many vaccinated participants had a recent history of immunosuppression and cancer diagnosis as unvaccinated participants (9.8% vs 6.7% and 25% vs 15.8%, respectively) ( Table 1 and Appendix Table 1, available online). We report the variables' missingness by vaccination status in Appendix Table 2 (available online).
Over 62% of the study population lived in the 4 Michigan counties surrounding MM (Washtenaw, Wayne, Livingston, and Oakland), whereas 5.7% lived outside of Michigan. Although Detroit resides in Wayne County, the population of Washtenaw County, where MM is located and most individuals lived, had over 72.6% vaccinated ( Figure 1 and Appendix Table 3, available online).

Rates of Infection and Severe COVID-19 Outcomes Among Unvaccinated and Vaccinated Individuals
Unvaccinated individuals were significantly more likely to test positive for SARS-CoV-2 infection (12.1%) than vaccinated individuals (4.7%). The positivity rate among vaccinated individuals reached a valley with low community transmission in Q2 (Q1: 1.3% and Q2: 1.1%) and surged to 3.2% in Q3 and to 6.7% in Q4 (Table 2). In contrast, the positivity rate among unvaccinated individuals was lowest in Q1 (7.6%), increased to 11.8% in Q2, and decreased to 7.9% in Q3, before reaching a peak of 19.3% in Q4.
The overall rate of the severe disease, a composite of COVID-19-related hospitalization, ICU admission, or  Table 2).

Vaccine Effectiveness
In Figure 2, we present VE estimates for the overall cohort (using data only from the latest calendar quarter in which each participant was tested) and time-stratified cohort (Q1, Q2, Q3, and Q4). Because the VE estimates were reasonably consistent across adjustments, for example, in the overall cohort, VE against SARS-CoV-2 infection (VE against infection) ranged between 59.4 and 62.1% (Appendix Table 6, available online), we only discuss the estimates from the most conservative setting (Adjustment 5, which included all the covariates mentioned earlier) in this study. Results from all models are listed in the Appendix Material (Appendix Tables 6 and 7 Figure 2 and Appendix Table 7, available online).
We also performed a sensitivity analysis using only data on individuals who received primary care at MM and thus were more likely to have documented vaccination data. In this analysis, estimates were substantially   Figure 2B, available online).

Comparing Vaccine Effectiveness Between Pfizer-BioNTech and Moderna
Because most of the 74,060 fully vaccinated individuals completed a vaccine series with either Pfizer-BioNTech (n=45,168; 61.0%) or Moderna (n=25,267; 34.1%), we compared the effectiveness of the 2 vaccines. To address that the Pfizer-BioNTech vaccine was available before Moderna, we stratified the analysis on the basis of whether the vaccination occurred <3 months or ≥3 months before the COVID-19 test (Figure 4).  Figure 5, available online).

Factors Affecting Vaccine Effectiveness
Overall, we found that VE against infection ranged between 57.1 and 65.2% across all strata but was lower for   Table 8, available online). In Figure 5, we found that compared to the trend among those aged <50 years, VE shows a decreasing trend among those aged >65 years against both infection and severe outcomes, but the differences were not statistically significant.
In addition, we observed decreased VE against infection in individuals living in neighborhoods with increasing NDI quartiles compared with that in those in the lowest NDI quartile (p Int. ≤0.023).

DISCUSSION
In a study of 170,487 adults tested for COVID-19 within Michigan, we found strong protection provided by Pfizer-BioNTech and Moderna vaccines against infection and severe disease. Booster doses increased effectiveness against both infection and severe disease. In our study, fully vaccinated individuals were less likely to test SARS-CoV-2 positive and were less likely to have the severe disease than unvaccinated individuals. We observed declines in estimated VE over time, indicating potential waning effectiveness, consequences of spreading of the more immune-evasive SARS-CoV-2 B.1.617.2 (Delta) and the B.1.1.159 (Omicron) variants. 37−39 Because uptake in booster shots increased in Q4, both infection and severe disease rates were lowest among boosted individuals. This evidence supports a booster's protection against infections and severe disease in the face of potential waning of VE and emerging virus variants.
Among individuals who received the vaccine within the past 3 months, VE against infection tended to be slightly higher for those who received the Moderna vaccine, and VE against severe outcomes was moderately high for both vaccines. However, the overlapping CIs suggest similar vaccine protection or the need for a larger sample size to tease out the vaccines' performance.
Our results are largely consistent with previously published findings, although differences among studies make direct comparison challenging. For example, some studies adjusted differently for potential confounders (age only, additional patient characteristics, or time since vaccination), [8][9][10]13,40 whereas others characterized the variant associated with each case through whole-genome sequencing. 11    to early 2021, VE estimates against hospitalization ranged around 87% (95% CI=55, 100) 8  Michigan offers an important location to study COVID-19 VE for several reasons. The southeast corner of the state, where this study is focused, has extensive socioeconomic disparities across its urban, suburban, and rural locations. Notably, Detroit, the most populous Black-majority city in the U.S., was severely affected by COVID-19 outbreaks early in the pandemic. Within the city, there were still substantial differences in COVID-19 vaccination coverage between Black (56%) and White (82%) populations in mid-2021. 41 Because there is substantial evidence that how effective the vaccine is could influence acceptance of the vaccine, 42 this study, reporting relatively high VE in this area, could be influential. However, we also note that VE, although not significantly different by race/ethnicity, might be lower among individuals living in neighborhoods with higher NDI. This suggests that more effort is needed to study protection from vaccines in more disadvantaged areas. Past studies of other respiratory infections have posited that differences in risk of infection across physical space could include housing overcrowding, limited access to health care, and higher rates of contact with others who are infected. 43 Future studies could sample within the community and not just among those with healthcare access. 44 Strengths of this study are the availability of detailed EHR data for study subjects, COVID-19 test and vaccination data from all the 4 quarters of 2021, and the large number of participants from a single academic medical center in Michigan. This allows us to adjust for potential confounding by patient characteristics, including socioeconomic and demographic characteristics, and relevant medical conditions, including previous COVID-19 testing results and receipt of a booster dose. Furthermore, we considered multiple outcomes of interest and assessed variation in VE by different demographic, clinical, and other factors. The availability of potential confounding patient characteristics allowed us to evaluate the potential modification of VE by patient subgroups. After correcting for multiple testing, none of the stratified analyses were significant. Differences before multiple testing corrections potentially indicate a stricter adherence to the vaccination series or additional protective measures about HCW. 45,46 There are some discrepancies in the literature on the effectiveness, with some literature suggesting that unvaccinated individuals who acquired immunity through a previous COVID-19 infection may benefit less from vaccination than people who were not previously infected. 47,48 However, there is a growing body of literature suggesting that COVID-19 vaccination is the best protection against infection and severe disease 49 whether or not one has already had COVID-19. 48,50−53 Limitations A potential weakness is any missing or incomplete information from the EHR, for example, variant sequencing, undocumented test results, past infections, vaccinations, hospitalizations, or mortality outcomes. Although we cannot test the exact completeness of the EHR vaccination record, for example, it is possible, although unlikely, that they may have been vaccinated elsewhere and that these records were not available, our sensitivity analysis examining MM primary care patients suggests the possibility that VE estimates in the full cohort might be influenced by incomplete vaccination documentation in the EHR. In this scenario, categorizing individuals into an unknown or unvaccinated status tends to produce conservative VE estimates (Appendix Figure 3, available online). Another explanation for these findings regards potential differences in health-seeking behaviors and therefore differential VE between patient groups. Results from the test-negative design are generalizable to individuals tested for COVID-19 and more straightforward with tests having similar sensitivity and specificity properties. 9,44,53 Still, these studies provide insight into the long-term effectiveness and durability of vaccine protection. 53 How to combine the RT-PCR test and the rapid antigen tests for VE studies in a statistically valid manner remains a question of interest in the future.

CONCLUSIONS
The study provides robust evidence for the ability of COVID-19 vaccines to protect against infection and severe disease. As new variants arise, continued observational studies will be necessary to assess the effectiveness of currently available vaccine regimens. Overall, these results provide evidence of the effectiveness of the vaccines over time and may encourage those who have not been vaccinated or not received a booster to consider doing so to prevent severe outcomes, including mortality.

ACKNOWLEDGMENTS
Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
This study was supported by the University of Michigan Precision Health Initiative, the University of Michigan Rogel Cancer Center, and the Michigan Institute of Data Science. BM's research was supported by Grant Number NSF DMS 1712933 from the National Science Foundation, and LGF's research was supported by Grant Number CA 046592 from the National Cancer Institute, NIH. Determination for exemption was made by the University of Michigan Medical School IRB (study identification HUM00180294).
Declarations of interest: none