{'fraction_non_alphanumeric': 0.025923054559550344, 'fraction_numerical': 0.016959007655780598, 'mean_word_length': 4.927340608845491, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Impact of Timing of and Adherence to Social Distancing Measures on COVID-19 Burden in the US: A Simulation Modeling Approach

Background Across the U.S., various social distancing measures were implemented to control COVID-19 pandemic. However, there is uncertainty in the effectiveness of such measures for specific regions with varying population demographics and different levels of adherence to social distancing. The objective of this paper is to determine the impact of social distancing measures in unique regions. Methods We developed COVid-19 Agent-based simulation Model (COVAM), an agent-based simulation model (ABM) that represents the social network and interactions among the people in a region considering population demographics, limited testing availability, imported infections from outside of the region, asymptomatic disease transmission, and adherence to social distancing measures. We adopted COVAM to represent COVID-19-associated events in Dane County, Wisconsin, Milwaukee metropolitan area, and New York City (NYC). We used COVAM to evaluate the impact of three different aspects of social distancing: 1) Adherence to social distancing measures; 2) timing of implementing social distancing; and 3) timing of easing social distancing. Results We found that the timing of social distancing and adherence level had a major effect on COVID-19 occurrence. For example, in NYC, implementing social distancing measures on March 5, 2020 instead of March 12, 2020 would have reduced the total number of confirmed cases from 191,984 to 43,968 as of May 30, whereas a 1-week delay in implementing such measures could have increased the number of confirmed cases to 1,299,420. Easing social distancing measures on June 1, 2020 instead of June 15, 2020 in NYC would increase the total number of confirmed cases from 275,587 to 379,858 as of July 31. Conclusion The timing of implementing social distancing measures, adherence to the measures, and timing of their easing have major effects on the number of COVID-19 cases. Primary Funding Source National Institute of Allergy and Infectious Diseases Institute

Appendix Table 9

COVID-19 progression in an individual
All individuals in COVAM are categorized into one of eight possible states representing an individual's COVID-19-related status ( Figure 1 in the main text): Susceptible (S), Exposed (E), Infected with mild to moderate symptoms that is detected (IM+), Infected with mild to moderate symptoms that is undetected (IM-), Infected with severe symptoms (IS), Infected with symptoms requiring critical care (IC), Recovered (R), and Dead (D). We adopt our states using the clinical states as described by the CDC and also introduced by the SEIR model of Hill. 3,8 A susceptible patient can be infected with COVID-19 only after exposure to SARS-CoV-2 by a contagious patient. If an individual is exposed to COVID-19, then s/he will move to the Exposed (E) state, and stay there during the incubation period of SARS-CoV-2. The individual may transmit the virus during the last several days of the incubation period, as evidenced by the literature. 9-12 After the incubation period is over, the individual becomes symptomatic with mild to moderate symptoms. There is a possibility (also representing the limited availability of COVID-19 testing) that the symptoms will be very mild that the individual will not be tested and the case will not be confirmed by health authorities, which is represented by the Infected with mild to moderate symptoms that is undetected (IM-) state. If the mild symptoms lead to testing that confirms the case, the patient moves to Infected with mild symptoms that is detected (IM+) state. Following the CDC's guidance and definition, patients in both IM+ and IM-experience mild symptoms. IM+ may also include mild pneumonia. In most cases, individuals in the IM+ and IM-states do not need to be hospitalized.
Patients in the IM+ and IM-states remain in these states the same amount of time and then they either move to Recovered (R) state or Infected with severe symptoms (IS) state. Following CDC's guidance, the IS state represents "dyspnea, hypoxia, or >50% lung involvement on imaging." 8 We assume that patients in the IS state require hospitalization. Patients in the IS state either recover and move to the Recovered (R) state or develop symptoms that require critical care, therefore move to the Infected with symptoms requiring critical care (IC) state. Following CDC's guidance, we assume patients in this state experience respiratory failure, shock, or multiorgan system dysfunction, therefore they need to be treated in an intensive care unit (ICU). 8 Some portion of these patients require to be cared with mechanical ventilation. Patients in the IC state either recover and move to the Recovered (R) state or die and transition to the Dead (D) state.
The primary mode of transmission for COVID-19 is human-to-human interaction, therefore we allow COVID-19 to be transmitted between individuals in E, IM+, IM-, IS, IC states and individuals in the S state.

Details of Parameter Estimation
We made the following assumptions to represent the probability of transmitting disease to susceptible patients by different patient classes.
-If the patient is a contagious exposed patient, then the probability of transmitting SARS-COV-2 to a susceptible patient is the same as that for when the patient is experiencing mild to moderate symptoms. -If the patient is experiencing severe symptoms, then the probability of transmitting SARS-COV-2 to a susceptible patient is a fraction of that for when the patient is experiencing mild to moderate symptoms. This is because, we assume that patients with severe symptoms require hospitalization and will stay in the hospital throughout this episode of the illness. We assumed that the probability of transmitting SARS-CoV-2 to a health care worker is 0 because due to heightened awareness of COVID-19 in the hospitals, patient to health care worker transmission possibility is very low. Furthermore, there is little impact of this parameter on the overall epidemic, therefore we made this simplifying assumption. -If the patient is experiencing critical symptoms, we assumed that the probability of transmitting SARS-COV-2 to a health care worker is 0. These patients are assumed to be receiving care in the ICU setting and there was not sufficient information to estimate its value. Furthermore, there is little impact of this parameter on the overall epidemic, therefore we made this simplifying assumption. With these assumptions, the key input for transmissibility is the probability of transmitting SARS-COV-2 from a patient experiencing mild to moderate symptoms to a susceptible individual when the patient is tested and confirmed positive with COVID-19. For this purpose, we followed a multi-step approach to adjust this parameter. We first started our parameter adjustment using the baseline estimate of R0 for COVID-19 as 2.6, which uses data from early days of the epidemic in China. 14 We then used our model's base input parameter values to find the probability of transmitting SARS-COV-2 from a patient experiencing mild to moderate symptoms to a susceptible individual such that the theoretical R0 value is 2.6, as listed next: there are 10 close contacts per person per day, duration for mild to moderate symptoms is 6 days, the transmission rate for patients experiencing severe or critical symptoms is 0. We found that when the probability of transmitting SARS-COV-2 from a patient experiencing mild to moderate symptoms to a susceptible individual is equal to 0.0418, the theoretical R0 value would be equal to 2.6 with these parameter settings.
Note that for this calculation, we did not account for the probability of transmitting disease from asymptomatic patients or from patients who experience mild to moderate symptoms and are not tested positive for COVID-19. This is because most models reporting R0 values for COVID-19 did not account for these transitions in their model development.
Once we set the values for both the probability of transmitting disease from asymptomatic patients and that from patients who experience mild to moderate symptoms and are not tested positive for COVID-19 equal to 0.0418, our model's theoretical R0 value has increased to 3.34 (without any interventions). This R0 is still within the range of reported R0 values, that vary between 1.5 and 6.5. 13,14 In particular, a recent study based on the COVID-19 epidemic in Italy reported an R0 value of 3.47 (when a SIR model is used) for the early days of the epidemic. 15 Similarly, a recent study estimated the median R0 value for Wuhan region as 5.7. 16 Therefore, we concluded that our transmission parameters are within the acceptable range.

Adherence to Social Distancing
An important input to COVAM is adherence to social distancing measures. As noted in the text, adherence to social distancing parameter represents several behaviors that reduce the transmissibility of SARS-CoV-2, including less frequent traveling, keeping at least 6-feet distancing during person-to-person interactions as well as frequent hand washing and wearing masks. COVAM does not have the ability to differentiate such behaviors from less frequent traveling. Our primary source for this parameter is traveling distance and keeping at least 6-feet distancing during person-to-person interactions. There are three main sources for this parameter: -Google's COVID-19 Community Mobility Reports 19 -Unacast's Social Distancing Scoreboard 17 -University of Wisconsin-Madison's Geospatial Data Science Lab 18 All three sources utilize cell phone data to estimate how the mobility of a community changed for each state and county in the US over time, however, they differ in the measurement of the mobility. For example, Google Mobility Report provides the data on the percentage change in the number of visits to different types of destinations including retail & recreation, grocery stores, parks, transit stations, and workplace. 19 For example, for NYC, the data from Google website (https://www.google.com/covid19/mobility/) shows significant increase in adherence to social distancing measures over time especially in late March and April. Unacast, on the other hand, reports three metrics to measure the efficacy of social distancing measures: change in average mobility (based on distance traveled), change in non-essential visits, and difference in encounter density. 17 Since all of these sources report different metrics and values for adherence to social distancing, we estimated the input values (as reported in Table 2 in the main text) using calibration. Our calibration procedure evaluated the adherence inputs that are higher than reported mobility values by these three different sources. This is because, as explained before, while our adherence parameter is a proxy for individuals' behaviors that reduce the transmission of the disease such as wearing masks and frequent hand washing, none of the data sources on mobility reported such behaviors. Instead, they only report data related to the reduction in the frequency of daily contacts. Our calibration procedure selected the adherence to social distancing values that allowed COVAM to match observed number of cases over time. Although COVAM has the ability to adjust this parameter on a daily basis, we changed the value of this parameter infrequently to prevent overfitting as explained below.

Calibration and Validation
Several model input parameters involve a high level of certainty, including disease transmission rates, probability of testing for COVID-19, and adherence to social distancing measures. As explained in the main text, we used a simple calibration procedure using earlier surveillance data from Dane County to fine-tune these parameters ( Table 1 in the main text). Adherence to social distancing measures changes on a daily basis for each region therefore, it is adjusted at regular time points as shown in Table 2 in the main text. Our modeling approach is very flexible therefore it is possible to match the observed number of cases almost perfectly by simply adjusting adherence to social distancing on a daily basis or changing the probability of testing for each day. However, we preferred to not perfectly match the observed data to prevent overfitting. For example, the model's predicted number of cases in the early days of the pandemic in Dane County are higher than observed number of cases, which is most likely due to the very limited testing in the early days. Considering a smaller probability of testing in early days would make the model match the observed data perfectly.

COVAM Mechanics
COVAM was coded in C++ for faster computational times and flexibility. Each replication of the simulation can last from a few seconds to an hour on a standalone desktop PC depending on the population size and number of infections. Although COVAM currently represents simple behaviors for the agents, it can easily be used to represent more complex behaviors by the agents. For example, the model is able to define the adherence levels of the individuals to the outcomes associated with COVID-19, i.e. when there are many cases/deaths in a week, individuals could become significantly more compliant with the social distancing measures.

Comparison of COVAM to other models
Given the lack of data on SARS-CoV-2, health systems and public health officials rely on mathematical models for policy decisions such as school closures and the extent and timing of social distancing measures. Most commonly used models compartmentalize the transmission and natural history of SARS-CoV-2 infection. Examples include the susceptible-infective-recovered (SIR) model and the susceptible-exposed-infective-recovered (SEIR) model, which extends SIR model by including a compartment representing the latent period. 43 After the sudden rise in COVID-19 cases in the U.S., several models specific to COVID-19 were quickly developed and made available online. 3-5 Some of the models provide both U.S. and state-specific prediction. 5 However, given vast within-state variability in transmission dynamics and adherence to social distancing, these models are severely limited in their applicability to local settings. 6 Some limitations include the assumption of a closed population, the inability to represent dynamic social distancing measures with different adherence levels accurately, and the inability to account for the effect of limited testing capacity on the number of confirmed cases. The major strength of our work is a simple and flexible model that incorporates similarities to existing compartmental models, but allows additional granularity when simulating critical features, thus capable of addressing complex policy questions. For example, COVAM can be used to evaluate the impact of increasing testing capacity on COVID-19 control as well as when it would be optimal to implement contact tracing for containing COVID-19 as it explicitly tracks individuals. As the COVID-19 epidemic has spread throughout the world, estimates of disease transmissibility in terms of basic reproduction number (R0) in subpopulations have varied widely. 13,14 Another strength of our ABM approach is that there is no static R0 input. Instead, the model directly models daily interactions and chance of viral transmission at each interaction, allowing these parameters to be varied independently at any point in time, and stratified by age group or other demographic characteristic. This allows evaluation of the impact of specific policy decisions on an epidemic.

Appendix B. Model Validation Results
In this section, we present the results of the model validation for the base inputs. All model inputs except adherence to social distancing were fixed before May 15, 2020. Adherence to social distancing measures changes on a daily basis for each region therefore, it is adjusted afterwards as described in the text and shown in Table 2 in the main text. The input parameter of adherence to social distancing measures was fixed as of May 15, 2020 for NYC and May 26, 2020 for Dane County and Milwaukee and the model results were compared to observed number of cases over time after this date. Figure 2 in the main text presents the results of this validation experiment. All results in this section are using the average values of 100 replications. Because our computational experiments use simulated data, standard errors could be minimized by simply increasing the number of replications. We used 100 replications to obtain stable estimates around mean parameter value and small standard errors.
In this set of validation experiments, COVAM's predictions were most accurate for the NYC, followed by Milwaukee and then Dane County. This is primarily due to a larger number of cases in NYC. Because the number of cases is relatively low in Dane County, it is likely that small events lead to major changes in the number of cases. For example, the outbreak in Dane County jail may have contributed to the larger number of cases in the week of April 24, 2020 whereas community spread is not affected with this event. 44 Our modeling approach is very flexible therefore it is possible to match the observed number of cases almost perfectly by simply adjusting adherence to social distancing on a daily basis or changing the probability of testing for each day. However, as explained in the main text, we preferred to not perfectly match the observed data to prevent overfitting. For example, the model's predicted number of cases in the early days of the pandemic in Dane County are higher than observed number of cases, which is most likely due to the very limited testing in the early days. Considering a smaller probability of testing in early days would make the model match the observed data perfectly.

Appendix C. Additional Computational Experiments
Appendix

Appendix D. Sensitivity Analyses
We conducted a sensitivity analysis on two parameters using NYC data: probability of testing and disease transmission. We changed the probability of testing from a baseline estimate of 75% to 25% and 50% and adjusted only the parameter on adherence to social distancing measures via calibration as described in the text. Namely, we compared the model predictions to observed number of cases in NYC by simply changing only adherence to social distancing measures. Our calibration resulted in the following estimates for the adherence to social distancing for each probability of testing value: • When the probability of testing is equal to 50%, adherence level is equal to 0% between March 4 and March 11; adherence level is increasing linearly from 0% to 80% between March 12 and March 28; and adherence level is at 90% after March 29. • When the probability of testing is equal to 25%, adherence level is equal to 0% between March 4 and March 11; adherence level is increasing linearly from 0% to 75% between March 12 and March 27; adherence level is equal to 90% between March 28 and April 20; and adherence level is at 85% after April 21. We did a similar sensitivity analysis on transmission rates. We halved the probability of transmission from a patient with mild to moderate symptoms (replaced the baseline estimate of 0.0418 with 0.0209) and recalibrated several model parameters including probability of testing, number of daily contacts, number of initial infections, and number of imported cases into NYC. We did not adjust the adherence to social distancing input for this sensitivity analysis. Our calibration resulted in the following values for these input parameters: -Baseline probability of testing: 50% -Number of initial infections: 1600 compared to the baseline estimate of 16, reflecting the theory that the pandemic in NYC has started much earlier than thought 45 -Number of external infections per day: 240 imported infections/day compared to the baseline estimate of 160 imported infections/day -Adherence level is equal to 0% between March 4 and March 11; adherence level is increasing linearly from 0% to 80% between March 12 and March 20; adherence level is equal to 80% between March 21 and April 28; and adherence level is at 90% after April 29. Below, we first present the results of the validation, which show that COVAM was able to replicate observed cumulative number of COVID-19 cases using the new set of social distancing adherence parameters for different values for probability of testing as well as when the rate of transmission is halved. We then present the results of our experiments.