{'fraction_non_alphanumeric': 0.03360013001259497, 'fraction_numerical': 0.020639499451509363, 'mean_word_length': 4.689782709200185, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Quantifying the effect of remdesivir in rhesus macaques infected with SARS-CoV-2

The world is in the midst of a pandemic caused by a novel coronavirus and is desperately searching for possible treatments. The antiviral remdesivir has shown some effectiveness against SARS-CoV-2 in vitro and in a recent animal study. We use data from a study of remdesivir in rhesus macaques to fit a viral kinetics model in an effort to determine the most appropriate mathematical descripton of the effect of remdesivir. We find statistically significant differences in the viral decay rate and use this to inform a possible mathematical formulation of the effect of remdesivir. Unfortunately, this model formulation suggests that the application of remdesivir will lengthen SARS-CoV-2 infections, putting into question its potential clinical benefit.


Introduction
A novel coronavirus, (SARS-CoV-2) has recently begun transmitting at a rapid pace around the world (Lippi et al., 2020). While many patients experience mild symptoms, the virus can lead to severe pneumonia and death Jiang et al., 2020). The rapid emergence of this virus has led to an urgent need to quickly find treatments that can improve patient outcomes for those who are severely ill (Martinez, 2020;Dhama et al., 2020).
Several broad spectrum antivirals are being investigated for treatment of SARS-CoV-2 (Yao et al., 2020;Elfiky, 2020;Du and Chen, 2019;Favalli et al., 2020), but one of the most promising to date is remdesivir (Ko et al., 2020), which was recently given emergency use authorization for treatment of SARS-CoV-2 by the FDA (Eastman et al., 2020). Remdesivir (GS-5734) was first developed as a possible treatment for Ebola (Warren et al., 2016), but was soon found to have broad antiviral effects on a number of different virus families (Lo et al., 2017), including coronaviruses (Sheahan et al., 2017;Brown et al., 2019;Parang et al., 2020). The mechanism of action of remdesivir in coronaviruses is interference with RNA-dependent RNA polymerase (RdRp) (Gordon et al., 2020a), specifically nsp12 polymerase in murine hepatitis virus (Agostini et al., 2018), nsp8 and nsp12 in Middle East Respiratory Syndrome virus (Gordon et al., 2020b), and nsp7 in the novel SARS-CoV-2 . The drug has shown some efficacy against SARS-CoV-2 in vitro (Choy et al., 2020;Wang et al., 2019;Pruijssers et al., 1101), as well as in animal studies (Williamson et al., 2020). There is some evidence of clinical benefit in patients (Holshue et al., 2020;Grein et al., 2020;Durante-Mangoni et al., 2020;Hillaker et al., 2020), although some adverse events have been noted (Durante-Mangoni et al., 2020). A number of clinical trials are underway (Eastman et al., 2020;chen Cao et al., 1016). One small observational study found clinical improvement of 68% of remdesivir-treated patients (Grein et al., 2020). Unfortunately, one completed clinical trial showed little benefit of remdesivir treatment in severely ill COVID patients, with an increase in adverse events , so there is still uncertainty about the benefit of remdesivir.
While in vitro, pre-clinical, and clinical studies are needed to definitively determine the effectiveness of remdesivir in treating COVID-19, mathematical modeling and computer simulations can help provide additional insight into how remdesivir interacts with the SARS-CoV-2 virus (Elfiky, 2020;Shannon et al., 2020;Zhang et al., 2020;Khan et al., 2020) and how that alters the viral kinetics . Mathematical modeling has been used to give guidance on timing of treatment of SARS-CoV-2 with other drugs (Gonçalves et al., 2020;Kim et al., 2020;Abuin et al., 2020;Chatterjee and Basir, 2020). Viral kinetics modeling has also previously been used to study treatment of other acute infectious diseases such as influenza (Dobrovolny et al., 2011;Melville et al., 2018;de Mello et al., 2018a), respiratory syncytial virus (RSV) (González-Parra and , Ebola (Madelain et al., 2018), andZika (de Mello et al., 2018b). This modeling has helped elucidate antiviral mechanisms (González-Parra and Dobrovolny, 2018;Cao and McCaw, 2015), quantify antiviral efficacy (Koizumi et al., 2017;Beggs and Dobrovolny, 2015), and investigate treatment timing (Gonçalves et al., 2020;Zhang et al., 2015), so this methodology might prove useful in helping develop tools to fight this pandemic. Two recent studies have used mathematical models to help explain why remdesivir seems to have different effects on nasal and lung titers  and to assess the effect of different treatment timings on viral time course .
In this paper, we use data from rhesus macaques infected with SARS-CoV-2 and treated with remdesivir to fit a viral kinetics model and determine the effect of remdesivir on the viral time course of SARS-CoV-2. We then used our results to investigate potential mathematical models for the effect of remdesivir. Our analysis finds that the only statistically significant difference between treated and control groups is slower viral decay caused by slower infected cell death in the remdesivir-treated group. This leads to a mathematical model of drug effect that predicts longer infection durations with remdesivir treatment.

Mathematical models
We use two models to help characterize the infection in untreated macaques as well as macaques treated with remdesivir. The first model is an empirical description of the viral time course, first presented in (Holder and Beauchemin, 2011). This model allows us to quantify different aspects of the viral titer curve although it does not provide any insight into the biological processes that might be affected by application of antiviral treatment. The model is given by the equation where g and d are the exponential growth and decay rates, respectively; V p is the peak viral titer; and t p is the time of viral titer peak. The second model we will use is a viral kinetics model consisting of ordinary differential equations (ODE). This model was originally used to describe influenza virus infections (Baccam et al., 2006), In the model, target cells, T, become infected, I, at rate β when they encounter virus, V. Infected cells produce virus at rate p and die at rate δ. Virus loses infectivity at a rate c. We have chosen not to include eclipse cells, which are cells that have been infected but are not yet producing virus, since the duration of the eclipse phase cannot be uniquely identified from viral titer measurements alone (Smith et al., 2010).
In addition to estimating model parameters through fitting of the model to data, we examine two additional quantities derived from model parameters. The basic reproduction number, Represents the average time between the virus being released from one cell and infecting the next.

Experimental data
Experimental data is taken from Williamson et al. (Williamson et al., 2020) who performed a study of the effectiveness of remdesivir treatment of SARS-CoV-2 in rhesus macaques. Briefly, 12 rhesus macaques were inoculated with a total dose of 2.6× 10 10 TCID 50 of the nCoV-WA1-2020 strain of SARS-CoV-2 via intranasal, oral, ocular and intratracheal routes. Nasal swabs were taken daily for seven days. 6 of the animals were treated with remdesivir initiated at 12 h post infection. These animals were given a loading dose of 10 mg/kg of remdesivir, followed by a daily maintenance dose of 5 mg/kg. The other six animals served as infected controls and were given an equal dose volume of a placebo solution on the same schedule. Two animals, one from each treatment group, were excluded from this study since we could not get accurate estimates for the decay rate of the viral load. We extracted the nasal swab data from Fig. 3A of (Williamson et al., 2020) using WebPlotDigitizer.

Data fitting
The models were fit to data by minimizing the sum of squared residuals (SSR), where n is the number of experimental data points, y i are the values of the experimental data points, and f t ( ) i are the model predictions at the times when experimental data were measured. A small SSR indicates a tight fit of the model to the experimental data. We used the nelder_-mead_min algorithm in Octave to find the minimum SSR. We fixed the initial number of target cells to 1, meaning that cells are measured relative to the total number of cells. The initial amount of virus is also set to 1 copies/mL. While the macaques were given an initial inoculum of 2.6× 10 10 TCID 50 , nasal swab measurement on day 0 returned values below the level of detection and was set to 1 copies/mL. We further set = c 10/d, as in (Gonçalves et al., 2020), since δ and c cannot be differentiated using viral titer data alone (Smith et al., 2010). To estimate the 95% confidence intervals for parameters, we perform 1000 bootstrapping replicates using the residuals from the best fit to generate new data sets, as described in (Efron and Tibshirani, 1986).

Statistical analysis
The goal of this study is to determine how remdesivir affects the viral kinetics of SARS-CoV-2. In order to identify statistically significant differences in parameter values between treated and untreated groups, we performed a Mann-Whitney (Wilcoxon rank-sum) test. We use the Mann-Whitney test since we cannot assume normal distributions for the parameters as is required for other statistical tests. When distributions are continuous, as they are in our case, the Mann-Whitney test can be interpreted as determining whether there is a significant difference in the medians of the two distributions. We consider p values less than 0.05 to be statistically significant.

Effect of remdesivir on viral kinetics
The experimental data, along with model fits to the data are shown in Fig. 1. The best fit parameters are given in Table 1 for the empirical  model and Table 2 for the viral kinetics model. Comparisons of the estimated parameter values for treated and control animals are shown in Fig. 2 for the empirical model, and in Fig. 3 for the viral kinetics model. Parameter correlation and distribution plots derived from bootstrapping are included in the supplemental material. While parameter distributions generally have a well-defined maximum, the parameter correlation plots show correlation among some of the parameters. In particular, for the empirical model, the time of peak and the growth rate are correlated in many animals and show an over-arching correlation when all animal bootstrapping results are plotted together (Fig. S11). For the viral kinetics model, we see correlations amongst all parameters that become more apparent when all animals are plotted together (Fig. S22). These correlations indicate that the parameters might not be uniquely identifiable.
For the empirical model, we find that the peak viral load (V p ) is slightly lower for treated animals than for untreated animals, although the difference is not statistically significant ( = p 0.35). There also appears to be a slight decrease in the time of viral peak for treated animals, but this is also not statistically significant ( = p 0.46). There is little change in the growth rate between the two groups ( = p 0.92), so the shorter time to viral peak is caused by the lower peak viral load of the treated group. The one statistically significant difference ( = p 0.028) we see between the groups is a difference in decay rates. Oddly, our results indicate a lower decay rate for the treated group meaning that the virus tends to linger for a longer period of time in treated animals than in untreated animals.
For the viral kinetics model, we see an increase in the infection rate in the treated animals, although this is not statistically significant ( = p 0.25). We also see a decrease in the production rate, although this is also not statistically significant (). Interestingly, the increase in the infection rate is roughly the same as the decrease in the production rate such that the infecting time remains largely the same between the two groups ( = p 0.75). The basic reproduction number, R 0 , increases for the treated group, although this is not statistically significant ( = p 0.12). This difference is caused by the statistically significant ( = p 0.028) difference in infected cell death rates between the two groups. Here, we again find that the death rate is lower in treated animals, consistent with the decreased viral decay rate found for the empirical model, but suggesting that infected cells live longer in treated animals than in untreated animals.

Mathematical modeling of remdesivir
The effect of a drug is added to viral kinetics models through the use of drug efficacy, ε, which is a number between 0 and 1 that represents the reduction in the value of particular parameter due to the drug. For example, the effect of oseltamivir on influenza infections is modeled by multiplying production rate by ( (Williamson et al., 2020). Experimental data is indicated by the circles, empirical model best fits are given by the solid red lines, and viral kinetics (ODE) model best fits and given by dashed blue lines. Animals 1-5 are untreated; animals 6-10 are treated.
100% effective drug ( = 1) yields no viral production. Our analysis of the application of remdesivir in rhesus macaques indicates that remdesivir decreases the infected cell death rate, which suggests that we apply the drug effect to δ. Fig. 4 (left) shows the predicted results of this assumption on viral load when different drug efficacies are applied to the model. We see that the peak viral load rises slightly as the drug efficacy increases, contrary to the findings of the previous section. The viral load also decays more slowly, consistent with the results of the previous section. This, however, seems like an ineffective drug -as more drug is applied (efficacy increases), the infection lasts longer.
An alternative possibility, suggested by our analysis of the previous section, is that remdesivir might have effects on multiple parameters. While not statistically significant, remdesivir appears to increase the infection rate and decrease the production rate. For simplicity, we assume that a particular dose of drug affects all three parameters equally, i.e. they are all changed by the same amount. In the case of the infection  rate, we divide β by (1 ) since we want the infection rate to increase as drug is applied. The effect of this assumption on viral load when different amounts of drug are applied is shown in Fig. 4 (right). In this case, increasing the amount of drug decreases the viral peak, consistent with results from the previous section. There is still a decrease in the decay rate of the viral titer with application of the drug.
The differences in predictions between the two drug models are shown in Fig. 5. The starkest difference is in how drug affects the viral peak, where one model assumption has the peak increasing as drug effect increases, while the other has viral peak decreasing as drug effect increases. The latter assumption is consistent with the decrease in viral peak noted in rhesus macaques. Unfortunately, both models predict that duration of the infection (defined as the time the viral titer is above the detection threshold of 10 0 copies/mL) will increase with application of drug. While the model where drug affects all three parameters predicts a slightly lower duration, both models are not suggestive of a clinically beneficial drug.
Another possibility is that the effect of remdesivir is modeled best by changing a parameter that is not included in our simple model. Infectious disease models typically include an eclipse phase after the

H.M. Dobrovolny
Virology 550 (2020) [61][62][63][64][65][66][67][68][69] virus has entered the cell, but before it starts actively producing virus. A previous study examining possible mathematical models for the effect of an RSV fusion inhibitor showed that modeling a drug by lengthening of the eclipse phase decreases the decay rate of virus (González-Parra and . We explore this possibility by adding the eclipse phase into the viral kinetics model, where E denotes cells in the eclipse phase and a new parameter k is the transition rate from eclipse to infectious. We set = k 3/d from the median value of k used in (Gonçalves et al., 2020). We model the effect of remdesivir as lengthening the eclipse duration, so we multiply k by (1 ). Results of this assumption are shown in Fig. 6. The predicted viral titer curves give the expected decrease in viral decay rate as well as a decreasing peak viral load as the amount of drug increases. Unfortunately, the time of viral peak is also delayed with increasing amount of drug, an effect not observed in our analysis.

Discussion
In this paper, we analyzed remdesivir treatment of rhesus macaques using an empirical description of the viral titer curve as well as a viral kinetics model. We found a decrease in peak viral load, a decrease in time of peak, an increase in infection rate, and a decrease in production rate in remdesivir-treated macaques as compared to control, although none of these were statistically significant. We found two statistically significant differences: the empirical model indicated that the viral curves of remdesivir-treated macaques decayed more slowly than the control group, which was consistent with the viral kinetics model finding that infected cell death rate was lower in remdesivir-treated macaques than in control. This is not consistent with results of a clinical trial that show similar viral decay rates in remdesivir-treated and untreated human patients . However, another mathematical modeling analysis noted that remdesivir potency was lower in the nasal passages than in the lungs and that this leads to increased viral titer in the nasal passages .
It should be noted that since we did not include an eclipse phase, and we fixed the viral decay rate, the only parameter in the viral kinetics model that could capture the decreased viral decay rate is cell death rate (Smith et al., 2010). Nonetheless, we used this result to explore a viral kinetics model that decreased the infected cell death rate, finding that it predicted both an increase in the peak viral load and lengthening of the duration of the infection. The increased peak viral load was inconsistent with our own analysis of remdesivir-treated macaques, as well as in vitro studies that show a decrease in the viral load (Choy et al., 2020;Wang et al., 2019;Pruijssers et al., 1101), although not necessarily measured at the peak. A mathematical model that incorporated a drug effect on infection rate and production rate, as well as cell death rate, fixed the problem of increasing viral load with application of drug, although it still predicted longer infections with application of drug. We explored the possibility of a drug that lengthens the eclipse phase, a parameter not originally included in the model, and found that it could capture the decreased decay rate as well as the decrease in viral peak. Unfortunately, this drug model suggested that remdesivir should also delay the time of peak, an effect we did not observe.
The mechanism of action of remdesivir can help in the formulation of appropriate mathematical models. Remdesivir interferes with replication of viral RNA (Gordon et al., 2020a;Yin et al., 2020). The virus originally enters the cell unimpeded, suggesting no change in the viral infection rate, but blocking viral RNA will impede production, so this is mechanistically the parameter that one would choose to apply the drug effect, and this is how other remdesivir modeling studies have modeled the effect of the drug . This is the parameter typically used to model the effect of oseltamivir in influenza (Dobrovolny et al., 2011;Baccam et al., 2006;HandelI et al., 2007;Palmer et al., 2017), so we know that this model predicts a decrease in Fig. 4. Predicted effect of a drug that only decreases infected cell death rate (left) or a drug that decreases infected cell death rate, increases infection rate, and decreases production rate (right). Median values of estimated parameters for the control group were used for the simulation. (right) Duration of the infection is similar for both drug models, although a drug that affects three parameters has a slightly lower duration. Median values of estimated parameters for the control group were used for the simulation.
the peak viral load and a lengthening of the infection duration as the time of peak is moved further out (Dobrovolny et al., 2011). It does not, however, change the viral decay rate. By hindering RNA replication, remdesivir could also be modeled by delaying the transition from the eclipse to the infectious phase. Applying the drug effect to the transition rate between phases does lead to a decrease in the decay rate as well as a slight decrease in viral peak, so this seems to be the most likely candidate for mathematically modeling the effect of remdesivir. It is also possible to construct even more detailed mathematical models that include intracellular processes (Zitzmann et al., 2020;Heldt et al., 2013) in order to apply the drug effect in a manner that more closely reflects the biological reality. Unfortunately, in order to properly parameterize a model that includes all these extra details, we need more than just viral titer measurements (Heldt et al., 2013;Miao et al., 2011), so it is not possible to test such models with the current data set.
We can compare the parameter values found here with other estimates of these parameters for SARS-CoV-2. There are two studies that have used modeling to estimate viral kinetics parameters in humans (Gonçalves et al., 2020;Kim et al., 2020;Hernandez-Vargas and Velasco-Hernandez, 2020). While it is impossible to compare any parameters that include viral units since they are not standardized, we can compare dimensionless parameters (R 0 ) and parameters that assess time scales (t inf ). for humans, which are all somewhat lower than the 12-69 found for our untreated group. The infecting time estimated for humans (~16-60 h) is much larger than the infecting time found here for untreated macaques (~1.0-2.0 h). Hernandez-Vargas et al. also found viral growth and decay rates where possible, finding growth rates of 3.16/d and 5.01/d for the two patients that had growth data available. These are both quite a bit smaller than our estimates that range between 11 and 32/d. The decay rates they found ranged from 0.39/d to 2.51/d; this trends lower than the 1.5/ d-3.2/d found here for the untreated group, although there is some overlap. Some of the differences found in viral kinetics could be due to differences in how the virus affects different species, but both studies are based on small numbers, so we cannot yet definitively draw that conclusion. Some of the estimated differences, however, are large enough that there should be caution in extrapolating results from this animal model to humans.
This study was largely limited by the data available for parameterizing the model. Viral titer measurements alone are not sufficient to uniquely determine the parameters of even our simple viral kinetics model (Smith et al., 2010;Miao et al., 2011). This means that we are not able to assess differences between treated and untreated groups for all the parameters -in this study we fixed the viral decay rate (c), so could not look for possible differences in viral decay rate. It also means that we cannot extend our model to include an eclipse phase and test a possible mathematical formulation of the effect of remdesivir that affects the transition rate from eclipse to infected. The number of animals used in the study is small, which means there is low power to detect statistically significant differences. We noted differences in the infection rate and the production rate between treated and control animals, but they were not statistically significant.

Declaration of competing interest
The authors whose names are listed immediately below certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers' bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript. Median values of estimated parameters for the control group were used for the simulation and k was set to 3/d.