{'fraction_non_alphanumeric': 0.038866678254823574, 'fraction_numerical': 0.015035633582478707, 'mean_word_length': 4.7531, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Design-Based Confidence Sequences for Anytime-valid Causal Inference

Many organizations run thousands of randomized experiments, or A/B tests, to statistically quantify and detect the impact of product changes. Analysts take these results to augment decision-making around deployment and investment opportunities, making the time it takes to detect an effect a key priority. Often, these experiments are conducted on customers arriving sequentially; however, the analysis is only performed at the end of the study. This is undesirable because strong effects can be detected before the end of the study, which is especially relevant for risk mitigation when the treatment effect is negative. Alternatively, analysts could perform hypotheses tests more frequently and stop the experiment when the estimated causal effect is statistically significant; this practice is often called"peeking."Unfortunately, peeking invalidates the statistical guarantees and quickly leads to a substantial uncontrolled type-1 error. Our paper provides valid confidence sequences from the design-based perspective, where we condition on the full set of potential outcomes and perform inference on the obtained sample. Our design-based confidence sequence accommodates a wide variety of sequential experiments in an assumption-light manner. In particular, we build confidence sequences for 1) the average treatment effect for different individuals arriving sequentially, 2) the reward mean difference in multi-arm bandit settings with adaptive treatment assignments, 3) the contemporaneous treatment effect for single time series experiment with potential carryover effects in the potential outcome, and 4) the average contemporaneous treatment effect in panel experiments. We further provide a variance reduction technique that incorporates modeling assumptions and covariates to reduce the confidence sequence width proportional to how well the analyst can predict the next outcome.


Introduction
Many organizations, especially digital native firms, run thousands of large-scale randomized experiments, often known as "A/B tests," to drive product innovation and augment managerial decision-making (Bojinov and Gupta, 2022a;Kohavi et al., 2013;Kohavi, Tang and Xu, 2020). Traditionally, randomized experiments run for a fixed time horizon during which experimental units (usually the customers) receive either a treatment (the new version) or a control (the standard offering). At the end of the experiment, an analyst determines the effectiveness of the treatment relative to the control on a business outcome by computing an estimated treatment effect and associated confidence interval typically through a -test. However, for many companies, the experimental units, or customers, arrives sequentially over time, and partial results are available throughout the study. In this setting, there are several reasons why picking a fixed time horizon to end the experiment and performing the analysis once is undesirable.
First, committing to a fixed time horizon is often difficult in practice. For example, power calculations to determine how many samples to collect are intractable for complex settings because it requires specifying the alternative hypothesis (Noordzij et al., 2010). Therefore, the analyst would often pick a natural but arbitrary unit of time, e.g., one week, to run the experiment (Lindon, Sanden and Shirikian, 2022). In this case, there are no statistical guarantees on the expected width of the confidence interval or the power. Thus, even if the confidence interval was very wide at the end of the experiment, the analyst is, unfortunately, unable to run the experiment longer due to post-selective inference issues (Taylor and Tibshirani, 2015). Furthermore, there are also practical constraints, e.g., resource and schedule changes, that make choosing a fixed time horizon more difficult.
Second, even if choosing a fixed time horizon is possible, it may be undesirable. For instance, if the treatment effect is large, the analyst could detect it with less data than they initially thought was necessary. This is particularly important as companies rely on experimentation to mitigate risk by detecting degradations or negative experiences quickly (Bojinov and Gupta, 2022b). At the same time, companies want the ability to precisely estimate the effect, even if it is relatively small. Unfortunately, it is impossible to satisfy both objectives with a fixed time horizon. If the sample size is small, then the experiment catches large negative effects early but is often under-powered to detect small effects. The aforementioned reasons show why it may be natural and desirable for the analyst to peek and make decisions as new data arrives.
However, it is well known that peeking at the data and taking any action will invalidate the subsequent statistical inference (Johari et al., 2017). To illustrate this issue, we show in Figure 1 the inflated type-1 error when an analyst performs a -test every time a new observation is observed, where each observation is independently drawn from a (0, 1). For example, when there are 500 observations the type-1 error is over 70% when = 0.10. The issue is further exacerbated if the data is dynamically and adaptively collected, i.e., if the next observation is sampled as a function of the previous observations.
To overcome this, companies and researchers have started using methods to compute confidence sequences to allow analysts to continously monitor the experiment (Lindon and Malek, 2020;Waudby-Smith and Ramdas, 2020a;Darling and Robbins, 1967;Wang and Ramdas, 2022;Howard et al., 2020). Confidence sequence are sequences of confidence intervals that are uniformly valid over time (often referred to as anytime-valid inference). Formally, a sequence set of confidence intervals { } =1 is a valid confidence sequence with type-1 error for the true treatment effect if holds for arbitrary data-dependent stopping rule , where is the final time of the experiment and can be determined in any data-dependent way. In words, Equation (1) allows the analyst to continuously perform inference through and also immediately stop the experiment as soon as the analyst is satisfied with the inference, e.g., when the confidence sequence do not cover zero or the width is sufficiently small. In other words, an algorithm could automatically terminate an experiment as soon as it detects a statistically significant effect and output the estimated treatment effect. Consequently, confidence sequences naturally allow large-scaled automation of thousands of randomized experiments. Traditionally, constructing confidence sequence requires technical regularity constraints or strong parametric assumptions on the observed outcome to construct supermartingales that could provide confidence sequences using Ville's inequality (reviewed in Section 3.1) (Howard and Ramdas, 2022;Waudby-Smith and Ramdas, 2020a;Waudby-Smith and Ramdas, 2020b). To overcome this limitation, we provide an extension of the anytime-valid inference paradigm to the design-based framework. A design-based approach conditions on the full set of potential outcomes, shifting the emphasis to the treatment assignment as opposed to the observed response. This approach has a long history dating back to Fisher and Neyman but has seen a resurgence in popularity as it allows us to perform inference on the obtained sample and handle complicated settings such as interference in an assumption-light manner (Fisher, 1935;Neyman, Iwaszkiewicz and Kolodziejczyk, 1935;Bojinov and Shephard, 2019;Ham, Imai and Janson, 2022;Rubin, 1974;Holland, 1986;Imbens and Rubin, 2015;Ding, Feller and Miratrix, 2016;Basse and Airoldi, 2017). See Abadie et al. (2020) for a review and comparison between design-based and sampling-based inference in randomized experiments.
Our work has several significant contributions. The main contribution of this paper is that we formally introduce the design-based approach to any-time valid inference that allows us to tackle a wide variety of problems. We build valid design-based confidence sequences for the average treatment effect for individuals (Imbens and Rubin, 2015), the reward mean difference in the multi-arm bandit setting (Sutton and Barto, 2018), the contemporaneous treatment effect for a single time series experiment with carryover effects, and the average contemporaneous treatment effect for panel data settings for individuals that are also observed across time (Bojinov and Shephard, 2019). Our design-based approach allows us to naturally relax any independence or distributional assumptions required for the data, allowing both the treatment assignment to be dynamically sampled and the response to be arbitrarily dependent on past assignments. Furthermore, our method allows researchers to incorporate any available covariates or modeling assumptions to reduce the confidence sequence width proportional to how well the analyst can predict the next outcome. To the best of our knowledge, our work is the first that formally introduces the design-based approach to confidence sequences for a wide range of problems, allowing close to assumption-free constructions of confidence sequences for various treatment effects without any technical assumptions on the distribution of our outcomes or explicit constructions of martingales.
Although our work is the first that introduces the design-based framework to sequential tests, anytimevalid inference was introduced as early as 1940s by Wald (Wald, 1945). Ever since, there has been substantial progress in building confidence sequences for a large number of problems with only minor regularity assumptions (Howard et al., 2020;Waudby-Smith et al., 2021). Our work builds upon the aforementioned works by incorporating the design-based approach to confidence sequences.
Our outline of the paper is as follows. In Section 2, we introduce the design-based framework and the relevant causal estimands and estimators for the most classical and simple setting, where the goal is to infer the average treatment effect from individuals. In Section 3, we first give a brief overview of how confidence sequences are constructed via martingales. Then, we propose an exact confidence sequence that is valid for general settings but depends on potentially unknown parameters of the data-generating process. Consequently, Section 4 introduces an improved design-based asymptotic confidence sequence. In Section 4.2, we introduce confidence sequences in the simplest case for the average treatment effect of individuals with independent data. In Section 4.3 we extend the results to allow fully dynamic and adaptive treatment assignments, allowing sequential testing in the multi-arm bandit literature. Section 5.1-5.2 generalizes the results to a time series setting with potential carryover effects and the panel data setting with units observed over periods. We also provide illustrative examples of using our design-based confidence sequence for each of the aforementioned setting. Section 6 proposes an important variance reduction technique by incorporating prior information or covariates. We end with simulations in Section 7 and a discussion in Section 8 of why the design-based framework is attractive for this setting.
We purposefully present results in the simplest settings first before generalizing the results to more complicated settings to emphasize how our design-based confidence sequences can tackle a wide variety of problems in great generality. We also believe that separating each cases and providing an example brings clarity both notationally and conceptually for what the goal and causal estimand is for each setting. For example, if the analyst only wanted to build confidence sequences for a single time series experiment, then proposing a general theorem based on the panel data setting will only clutter notation and may lead to confusion.

Design-based causal inference
We begin by introducing classical causal estimands and estimators in a design-based framework. We observe independent samples of { , } =1 from users, where is the treatment assignment of the th individual and is the observed outcome for unit . Each user has a pair of potential outcomes { (1), (0)}, where we focus on the binary treatment assignments for simplicity, but our results generalize to treatments with multiple levels. A design-based or finite-population approach to causal inference conditions on the potential outcome, i.e., { (1), (0)} is fixed for every individual . To make this formal, we denote  , as the sigma-algebra that contains all pairs of potential outcomes { (1), (0)} =1 and all observed independent data up to the th user { , } =1 conditional on the pairs of potential outcomes. This approach makes the observed response = (1) + (1 − ) (0) random only through the randomness of , where we have assumed the no-interference assumption of potential outcomes, i.e., one unit's treatment assignment does not impact another unit's outcome (Imbens and Rubin, 2015;Cox, 1958). In Section 4.3 and 5.1, we relax both the independence and no-interference assumptions for the multi-arm bandit and time series settings. As a reminder, the design-based approach crucially avoids making any assumptions about the distribution of the potential outcomes and solely leverages the randomness of for inference.
Our goal is to estimate the average treatment effect for the individuals, where is defined the following way.
where denotes our finite-sample population of interest. In a sequential experimental setting, changes as more data arrives (further formalized in Theorem 3.2). Although (1) − (0) is never jointly observed, we use the inverse propensity score estimator (Horvitz and Thompson, 1952;Imbens and Rubin, 2015)̂ that is unbiased for (1) − (0), wherê and ( ) ∶= Pr( = ). To ensure that̂ is well defined, we make the following positivity assumption.
In words, Assumption 1 states that every individual has some positive probability of receiving the treatment. Lastly, classical results show that the average of̂ is an unbiased estimator of (1) − (0), providing also a simple estimate of the (upper bound) of the variance. We state this in the following lemma. Furthermore, the upper bound of the variance has the following unbiased estimator The proof is provided in Appendix A . The variance estimate is an upper bound because the actual variance term contains the product (1) (0), which are never jointly observed. Thus, we use the following inequality 2 + 2 ≥ 2 for any , ∈ ℝ to obtain this upper bound.

Design-based confidence sequences
In this section, we first give some theoretical background on how confidence sequences are constructed through martingales. Then we introduce the first design-based exact confidence sequence in a general setting that has practical issues, motivating the need for a better confidence sequence.

Confidence sequence construction through martingales
We first introduce how confidence sequences are constructed through martingales. Equation (1) requires that the confidence sequence contains the target parameter at every time. On one hand, this can be viewed as a multiple-testing problem, where we desire to perform a hypothesis test at every time new data arrives while controlling for type-1 error. Although any naive approach, e.g., Bonferroni correction, will technically lead to valid type-1 error guarantees, such an approach leads to uncontrolled type-2 error since → 0 as time grows. Thus, the state of the art approaches have used martingale constructions that automatically give a time-uniform guarantee through Ville's maximal inequality (Ville, 1939).
Lemma 3.1 (Ville's Maximal Inequality). Let be a non-negative supermartingale with respect to a filtration  . Then, In particular, if the initial value 0 = 1, then Lemma 3.1 reduces exactly to the desired uniform type-1 error guarantee in Equation (1). Thus, most constructions of confidence sequences rely on constructing a non-negative supermartingale and applying Lemma 3.1 to achieve the time-uniform guarantee, e.g., (Lindon and Malek, 2020;Howard et al., 2020;Waudby-Smith and Ramdas, 2020a).

Design-based exact confidence sequence
We now present our first design-based confidence sequence for by utilizing the empirical Bernstein inequalities for bounded potential outcome (Howard et al., 2020).  We remark that can be extreme to make this assumption hold, which is widely used to satisfy regularity conditions used in design-based inference (Bojinov and Shephard, 2019;Lei and Ding, 2020). In the finitepopulation framework, it is likely implausible for any units to have infinite potential outcome. Furthermore, Assumption 2 is about the realized potential outcome. For example, if user's potential outcome was generated from a (0, 1) distribution, an unbounded distribution, each of the user's realized potential outcome is still bounded. Therefore, we view Assumption 2 as a mild regularity condition. Given these assumptions, we can state our first design-based exact confidence sequence. 1 With a slight abuse of notation we also use (the population size) as a stopping time because once the experiment is terminated at "time" , the confidence sequence infers the average treatment effect of the individuals in the available data, making it the "population". Furthermore, because is a data-dependent stopping time, we require that it is formally a well-defined stopping time, i.e., a measurable function dependent on the current and previous data (not on the future). The proof is provided in Appendix B. As mentioned in Section 3.1, the key part of the proof is that we show forms a supermartingale. The rest follows from algebraically manipulating the supermartingale after applying Lemma 3.1. There are three practical limitations to Theorem 3.2. First, the confidence sequence scales with , which is dependent on both the observed and missing potential outcomes. Although Assumption 2 can be seen as a mild regularity condition as finite-sample units likely do not have infinite potential outcomes, analysts often do not know this extreme value, except in special cases such as binary outcomes. This issue is further exacerbated if there exists one extreme potential outcome.
Second, in the spirit of a sequential test, the analyst may desire to change ( ) as more units enter. However, Theorem 3.2 requires to be determined before the start of the experiment, thus restricting the treatment assignment probabilities to a pre-specified range. Although this is common practice for many experiments, the confidence width further scales inversely with . Third, the confidence width is of order approximately ( ∕ ). ∕ likely does not shrink asymptotically ( → ∞) to zero unless grows sub-linearly (̂ vanishes to zero) or there are stronger assumptions on the potential outcomes. This issue is not new as (Howard et al., 2020) dedicate Section 3 of their paper to solve this issue under additional conditions. We choose to present it under general conditions because it is still useful for special cases (see Example 1) and emits a closed-form design-based confidence sequence that is easy to implement. Nevertheless, we show in Appendix C how to leverage a mixture distribution with the truncated gamma distribution to build another confidence sequence with order roughly ( log ∕ ), which does not have a closed-form and requires root-solving algorithms.
In the next section, we explore an alternative approach that solves all the aforementioned issues. Before presenting our alternative, we first give an example to illustrate this confidence sequence in practice. All examples presented in this paper serve to aid the reader's understanding and a more comprehensive simulation study is presented in Section 7.
Example 1 (Online Experimentation). Suppose individuals visit a website sequentially over time. When the page loads, each person is randomly assigned with probability 1∕2 to the original version "A" (control) or the new version "B" (treatment). Suppose we are interested in a binary outcome that tracks if the user clicks on the sign-up page. Then, in the notation from Theorem 3.2, = 1 and = 1∕0.5 = 2. Finally, suppose the ground truth is that each user has a 0.15 and 0.05 chance of clicking on the sign-up button for versions "A" and "B," respectively; mathematically, Pr( (0) = 1) = 0.15 and Pr( (1)) = 0.05 for all . The average treatment effect is then -0.1. The left and right panels of Figure 2 show the confidence sequence from applying Theorem 3.2 on one simulated experimental data in the aforementioned setting for = 500 and = 10, 000 units, respectively. The left panel shows that the analyst would likely stop at the 210 th individual because the confidence sequence statistically significantly shows a negative treatment effect (the first time the red contours do not overlap the black dotted line), mitigating further harm. However, the right panel shows that the confidence width does not shrink despite a large = 10, 000, illustrating the aforementioned limitations.

Design-based asymptotic confidence sequences
We now improve the design-based exact confidence sequence introduced in Section 3.2 by constructing a design-based asymptotically valid confidence sequences. Asymptotic confidence sequences were first introduced by (Waudby-Smith et al., 2021), and we extend their work to the design-based framework. Before providing the main result of this paper, we first define an asymptotically valid confidence sequence.

Asymptotic confidence sequence
Informally, asymptotic confidence sequences are valid confidence sequences after a "sufficiently large" time. Although this may worry practitioners since confidence sequences, by definition, should have valid coverage at all times (including early times), we show through simulations in Section 7 that the empirical coverage remains robust in practice because the confidence sequence width for early times is wide in part due to our upper bound variance estimator.
Definition 4.1 (Asymptotic Confidence Sequences). We say that (̂ ± ) is a two-sided (1 − ) asymptotic confidence sequence for a target parameter if there exists a non-asymptotic confidence sequence (̂ ±̃ ) for such that̃ Furthermore, we say has an approximation rate if̃ − = . . ( ), where can be interpreted as how fast the approximation error̃ − is decreasing.
To readers familiar with asymptotic confidence intervals, the above definition may be puzzling. To give some intuition, the idea of "couplings" has been used in the literature of strong approximations and invariance principals (Einmahl, 1987;Komlos, Major and Tusnády, 1976) to formally define asymptotic confidence intervals. This literature defines a (1 − ) asymptotic confidence interval if there exists a nonasymptotic (unknown) confidence interval centered at the same statistic such that the difference between this non-asymptotic and asymptotic confidence interval is negligible asymptotically. Equation (4) captures the same notion except we replace all convergence statement with almost-sure convergence to satisfy the time uniform guarantee required of confidence sequences. 2 Throughout the rest of the paper (including the appendix), we omit subscript "a.s." from . . (.) and . . (.) to simplify notation.

Design-based asymptotic confidence sequence for the ATE
We now state one of the main result of our paper, namely that we can construct asymptotically valid confidence sequences for usinĝ and our estimated variancê 2 . Before stating the theorem, we require one additional assumption so that our variance is well-behaved, i.e., does not vanish in the limit.
We say Assumption 3 holds if Informally, Assumption 3 is satisfied as long as the potential outcomes do not vanish to zero as time grows. Conversely, one way Assumption 3 does not hold is if the response of all the users arriving over time suddenly vanish to zero after a certain time. We believe this is highly unlikely in practice, thus we also view Assumption 3 as a mild regularity assumption.
Theorem 4.1 (Design-based Asymptotic Confidence Sequence for the ATE). Assume the same setting as that in Theorem 3.2, where additionally Assumption 3 is satisfied. Then, forms a valid (1 − ) asymptotic confidence sequence for the average treatment effect with approximation rate √̃ log̃ ∕ for any pre-specified constant > 0.
The proof is in Appendix D. The key difference between this result and Theorem 3.2 is that the confidence width scales approximately (ignoring log terms) as ( √ ∕ ) (similar to a -test) as opposed to ( ∕ ). Furthermore, the confidence width has no expressions related to or , thus making it a fully general result. Lastly, is typically chosen by the analyst to minimize the confidence width at a certain fixed time.
For all examples and simulations, we choose so that the width is minimized at time 10 or, equivalently, the 10 th individual. We do this because the confidence sequence width is largest at early times, and the choice of becomes insignificant as more data arrives. We additionally show a closed-form expression to calculate in Appendix E. For practitioners who do not want to calculate , we recommend using the same we do throughout the paper, i.e, ≈ 0.77. We now return to Example 1 to illustrate Theorem 4.1.
Example 2 (Online Experimentation Continued). Suppose the same setting as that presented in Example 1. Figure 3 shows the confidence sequence from applying Theorem 4.1 on the same simulated experimental data in the aforementioned setting for = 500 and = 10, 000 units. The left panel shows the analyst would likely terminate the experiment at approximately the 200 th unit. Although the left panel of Figure 3 is not substantially different than the left panel of Figure 2 (largely because is not extreme due to the binary outcomes), the right panel shows that the confidence sequence width shrinks to zero unlike the confidence sequence in the right panel of Figure 2.

Dynamic updating and bandit setting
Although we presented Theorem 4.1 for independent Bernoulli treatment assignment, we can also construct confidence sequences when the treatment assignments are adaptive based on previous data. This now allows us to build confidence sequences for inferring "treatment" effects for bandit settings, where an agent is typically tasked to maximize reward and pull the next arm as a function of all the previous treatment assignments (Slivkins, 2019;Sutton and Barto, 2018).
To formalize this, we define the adaptive probability assignments the following way with the same positivity assumption.
Similar to Assumption 3, we assume the variance based on the new adaptive probabilities do not vanish.

Assumption 5 (None Vanishing Variance in Dynamic Settings
is replaced with the adaptive probability assignments. Then we assume that With these new definitions and assumptions, we directly extend Theorem 4.1 for the bandit setting with the following corollary that is proved in Appendix D. forms a valid (1 − ) asymptotic confidence sequence for the average treatment effect with approximation rate √̃ | −1 log̃ | −1 ∕ for any pre-specified constant > 0.
We remark that the confidence width in Corollary 4.1 has all the same benefits as those enjoyed by the confidence width in Theorem 4.1. For example, it does not depend on any hyperparameters of the data, e.g. , and the width also asymptotically shrinks to zero.
Example 3 (Two Arm Bandit). Suppose for simplicity the two-arm bandit problem, where an agent pulls either arm A (control) or arm B (treatment). Suppose the rewards under arms A and B have distributions (1, 1) and (2, 1), respectively. Consider the following adaptive probabilities for the th individual and | −1 (0) = 1 − | −1 (1), wherē 1, −1 is the sample mean of arm B using the obtained samples for = 1, 2, … , − 1 and̄ 0, −1 is defined similarly. In other words, the agent upweights the arm that produce a higher mean reward based on the sample means. Lastly, the agent does a fair coin flip for the first ten time periods (exploration period). Figure 4 shows that even under adaptively sampled data, our confidence sequence tightens to the desired truth. For this case, the agent would likely terminate the experiment at approximately = 150 because the reward from arm is statistically significantly higher than the reward from arm .

Extensions to time series experiments and panel data
So far, we have built confidence sequences in the classical setting where units or rewards are observed over time with potentially adaptive treatment assignments. In this section, we extend our confidence sequences in time series experiments where a single unit receives multiple treatments over time and there exist carryover effects (that is, past treatments impact current outcomes). Then we further extend to the panel data set, where we have time series experiments for all units. The panel data setting is especially common for many organizations where they randomly assign customers to treatment and control and observe them over time.

Time series experiments with carryover effects
We begin by denoting  , as the sigma-algebra that contains all pairs of potential outcomes and the observed data { , } −1 =1 up to time − 1 conditional on the pairs of potential outcomes. We purposefully change the notation to denote in the subscript so that readers can conceptually understand that we are now in a time series setting. In other words, the unit of interest are not necessarily individuals or rewards observed over time but a single time series unit.
In this settings, there are often strong carryover effects, i.e., the potential outcome is not only a function of its current treatment assignment but of the whole treatment assignment path ( 1 , 2 , … , ). For example, organizations, e.g., Uber and Lyft, use switchback experiments to employ a different algorithm ( ) at each time and the potential outcome may depend on the previous treatment paths (Bojinov, Simchi-levi and Zhao, 2020;Chiara Farronato, 2018). As a second example, the Uber ride sharing platform has a constantly evolving dynamic between drivers and customers, where the response is heavily time-dependent on the previous state of drivers and customers. In this setting, the assumption that previous versions of the treatment does not affect the current response is unlikely. Consequently, we weaken the no-interference assumption for potential outcomes and build confidence sequences for causal effects that account for carryover effects through a design-based approach.
For any random variable , we first denote 1∶ = ( 1 , 2 , … , ), thus 1∶ denotes the vector of treatment paths up to time and we write 1∶ as a realization of the random variable 1∶ . We allow our potential outcomes to be dependent on all the treatment assignment path by writing ( 1∶ ), where we assume that the potential outcome at time does not depend on future treatment assignments. Our observed outcome is = ( 1∶ ), where 1∶ is our observed treatment assignment path. We then denote the entire collection of potential outcomes up to time as where (•) = { ( 1∶ ) ∶ 1∶ ∈ {0, 1} } denotes the entire possible collection of potential outcome at time . As similarly done before, the design-based approach conditions on (•) for all . Unfortunately, the number of potential outcomes grow exponentially with , thus we focus on the contemporaneous causal effect that is a function of our observed treatment path.

Definition 5.1 (Contemporaneous Causal Effect).
( 1∶( −1) ) = ( 1∶( −1) , 1) − ( 1∶( −1) , 0), Definition 5.1 captures the contemporaneous treatment effect (CTE) had the unit received treatment at time . We specifically define the causal estimand as a function of our observed treatment path to show that our causal estimand changes as a function of our treatment path. Defining the treatment effect in this way is similar to focusing on the average effect on the treated, which is a widely accepted causal estimand (Imbens and Rubin, 2015) (see Section 3.3 of (Bojinov and Shephard, 2019) for further discussions of Definition 5.1).
Lemma 5.1 (Mean and variance). Under Assumption 1, we have that (̂ |  , −1 ) = ( 1∶( −1) ) and that Var(̂ | −1 |  , −1 ) ≤ (̂ | −1 |  , −1 ), where with a slight abuse of notation  , −1 is the sigma algebra containing all possible potential outcomes 1∶ (•) and observed data { , } −1 =1 up to time − 1. The proof is provided in Appendix A. Like Assumption 2, we similarly assume these new potential outcomes (with potential carryover effects) are bounded. The confidence sequence in Theorem 5.2 is identical to that in Corollary 4.1 except the confidence sequence covers the running mean of the contemporaneous treatment effect as opposed to the average treatment effect and the notation is defined with respect to as opposed to to conceptually illustrate the different settings. The proof is in Appendix D, where this proof proves all asymptotic design-based confidence sequences provided in this paper. For example, Theorem 4.1 can be recovered by assuming that all treatment assignments are independent and there are no carryover effects. Corollary 4.1 can also be recovered by assuming away carryover effects. We choose to present it this way so that readers can understand the wide variety of problems our confidence sequence can tackle. Lastly, although all examples thus far have considered stationary treatment effects, i.e., treatment effects that do not vary by time, our confidence sequences cover time varying causal effects. We demonstrate this in the following example.
Example 4 (Novelty Effect of Message Alerting Treatment). Suppose at each time the treatment is to alert the one user in our study with a message to engage with a product. Suppose the user's engagement increases substantially in the beginning based on the new treatment but the treatment effect diminishes over time (to zero) because the user has grown used to the treatment (often known as novelty effect). Furthermore, we assume that the message alerting treatment is only effective if the user did not receive a message alert at the previous time. Formally, we have that and ( −1 = 0, = 0) = ( −1 = 1, = 0) = ( −1 = 1, = 1) = (25, 10 2 ). We exaggerate the initial treatment effect to start at 500, which decreases with order 1∕ √ , to clearly show the time varying effect. Although real messaging alert treatments likely have more than lag-1 carryover effects, we use this simplified setting for illustration. Figure 5 shows that the confidence sequence uniformly covers the true time varying running mean of the contemporaneous treatment effect (in red) at all times. Although the analyst would reject the null effect at approximately = 200, the analyst would also see a diminishing time varying treatment effect had he/she suspected a novelty effect and continued running the experiment.

Setting and notation
The above theorems and results are in the context of time series experiment. In practice, many organizations also run time series experiments for units, where we observe multiple responses , for each unit = 1, 2, … , (Bojinov, Rambachan and Shephard, 2021). In this setting, we also observe the treatment assignment , for each unit across time , where as before , is a realization of this random variable and ,1∶ denotes the entire treatment path for unit until time . Although our above setting allows for any general carryover effects, in this setting we assume that each unit's potential outcome is only a function of its own treatment assignment path. More formally, Assumption 7 allows us to denote each unit's potential outcome as , ( ,1∶ ), where we again assume the potential outcome is also not a function of the future treatment assignments. As before, we denote the entire collection of potential outcome for all units at time as The new filtration  , , denotes the sigma algebra containing the information set of all treatment assignment ,1∶ and observed outcome , up to time for all units . Furthermore, the filtration always contains the entire potential outcome set ,1∶ (•). Similar to Assumptions 4 and 6, we assume all the potential outcomes are bounded and the treatment assignments are bounded away from zero or one, which we now state under one assumption. Assumption 8 allows the experimenter to adapt the treatment assignments for unit not only as a function of the previous unit 's treatment assignment and response but also all the other units' treatment assignment and response. Lastly, since we have multiple users , our causal estimand changes to the contemporaneous treatment effect averaged over units.

Aggregation
One naive, but powerful, approach to sequentially test for , ( 1∶ ,1∶( −1) ) is to simply "stack" or aggregate our data and pretend we have one single time series of observations. To illustrate this, consider our full panel data matrix

Units
where the rows denote time horizon and the columns denote the units. At each time , we observe the th column of the above matrix. The naive aggregation method concatenates all columns from the above matrix into one row and pretends the data comes from one time series .
With this aggregation, we observẽ = 1, 2, … , time points and apply Theorem 5.2 on the above aggregated time series. Because at each time we simultaneously observe observations, the width of the confidence sequence further decreases with . Since many organizations typically have large number of experimental units, the confidence sequence width can be small even at very early times and the asymptotics more credible. To formalize this, we further denote the aggregated causal estimates and variance estimates aŝ are the corresponding individual level counterparts of̂ | −1 and̂ | −1 , respectively. Finally, we also have the corresponding assumption for Assumption 5.

Assumption 9 (None Vanishing Variance for Panel Data Setting). Let Var(̂
, | −1 is equivalent to 2 | −1 (defined in Assumption 5) except it is defined for individual level potential outcomes and adaptive probability assignments. Then we assume that The proof is omitted because under the stated assumptions it is identical to that of Theorem 5.2 replacing time with̃ = 1, 2, … , and stacking the panel data into one time series. Theorem 5.3 shows that our confidence sequence width further decrease with 1∕ .
One advantage of the aggregation approach is that it can also account for units that enter the experiment at different times. Although for clarity we present the results when there are exactly the same units at each time , we lose no generality when the total number of units are different at each time , i.e., units can both be entering and leaving at any time point . In such a case the causal estimand will also consequently change according to the different units that enter and exit. We present an example using Theorem 5.3 in Example 5.

Variance reduction technique via proxy outcomes
In the previous sections, our confidence sequences were constructed with estimators that only leverage the observed treatment and outcomes. In practice, there may be available covariates or prior information that an analyst can leverage to further reduce the confidence sequence width through the use of proxy outcomes.
Note that, although we present our results in the time series and panel experiment setting, they immediately apply to the independent units setting studied in Section 4.
We remark that both the prediction̂ | −1 and the filtration are functions of all covariate information 1∶ , including covariate information not necessarily available at time . We present it this way to clarify two points. First, in non-time series settings like Example 1, the covariate information for all the users in the experiment may indeed be known before the beginning of the experiment. For example, for companies that conduct member-based experiments, the covariates are known and available to the analyst before the beginning of the study. In this case, conditioning on all the covariates of the members at any time accurately reflects the aforementioned setting. Second, there may be settings where covariates are evolving over time or the units in the experiment are not necessarily known in the beginning of the experiment, e.g., new member experiments. In such settings, the analyst still desires to use to construct the prediction̂ for . To allow this, we allow both the prediction and the filtration to condition on all , including the future, similar to how we always condition on all the potential outcomes. In practice, however, the analyst will typically only leverage the available covariates, namely 1∶ , for predicting at time .
Using this proxy outcome, our causal estimand in Definition 5.1 can be rewritten as, Using the above formulation, the corresponding estimator using the proxy outcome iŝ with a similar upper-bound estimate of the variance of 2 | −1 ∶= respectively. One can directly see that̂ | −1 is again unbiased for ( 1∶( −1) ) conditional on  , , −1 , where conditioning on all the 1∶ does not harm our inference becausê | −1 still remains a constant conditional on the filtration (hence it is crucial that the predictions are constructed based on the past data without using ). This allows the analyst to formally use the proxy outcomê | −1 to incorporate any machine learning algorithm or prior knowledge to reduce the variance. This reduction is proportional to how small { −̂ | −1 } 2 is, i.e., how well the analyst can use the prior data to predict the next response.
Furthermore, we can also allow treatment assignment probabilities | −1 ( ), defined in Assumption 4, to depend on the covariates since we always condition on  , , −1 . Lastly, we also require that the new variances with the proxy outcome do not disappear and that the proxy outcomes do not output infinity so that our new outcome −̂ | −1 remains bounded.
Assumption 10 (None Vanishing Variance with Proxy Outcomes). Let Var(̂ | −1 |  , , −1 ) ≤ 2 | −1 , where 2 | −1 is provided in Appendix A Equation (10) and similar to the expression in Equation (5) except we replace each potential outcome in the numerator with ( 1∶( −1) , •) −̂ | −1 . Then we assume that Further, we have that the predictions do not return infinity, i.e., |̂ | −1 | ≤ ′ for all and ′ ∈ ℝ. forms a valid (1− ) asymptotic confidence sequence for the running mean of the contemporaneous treatment effect̄ ( 1∶( −1) ) with approximation rate √̃ | −1 log̃ | −1 ∕ for any pre-specified constant > 0. The proof is omitted because the setting as well as the assumptions are identical to that of Theorem 5.2 except we replace | −1 with | −1 and introducinĝ | −1 is equivalent to changing the (fixed) potential outcome ( 1∶( −1) , •) to a new constant ( 1∶( −1) , •) −̂ | −1 . Since the entire proof always conditions on the filtration, the proof remains identical. However, this difference allows the confidence sequence in Theorem 6.1 to incorporate covariates and other prior information to potentially reduce the confidence sequence width. We further remark that our covariates can contain both pre-treatment covariates that do not evolve over time, e.g., user sex and race, browser and device type, etc., and time varying covariates that can evolve over time (even as a function of previous treatment assignments making it a post-treatment confounder). Additionally, even if there are no available covariates, one could still likely reduce variance by using the sample mean of 1∶( −1) for̂ | −1 , thus making Theorem 6.1 a useful practical extension for many cases.
Although we present Theorem 6.1 in the general time series setting with carryover effects, the same variance reduction technique also extends to non-time series setting in Theorem 4.1 and Corollary 4.1. In the aforementioned setting, Theorem 6.1 would be identical except all expressions with subscript are replaced with , allowing the proxy outcomê | −1 to instead predict the next user's response as a function of all the previous data and available covariate information.

Proxy outcomes in panel data setting
The variance reduction technique via proxy outcomes is most applicable in the panel data setting because we can leverage common information shared across users (as opposed to only one user) to make predictions for the next time point. Although generalizing the results in Section 6.1 to the panel data setting is straightforward, we formalize this for completeness.
For the panel data setting, the corresponding estimators using the proxy outcome arê Similar to Assumption 10, we have the following assumption.

Assumption 11 (None Vanishing Variance with Proxy Outcome Variances for Panel Data). Let Var(̂
, | −1 is equivalent to 2 | −1 (defined in Assumption 10) except for individual level potential outcomes and adaptive probability assignments. Then we assume that , | −1 . Further, we have that the predictions do not return infinity, i.e., |̂ , | −1 | ≤ ′ for all , and ′ ∈ ℝ. Comparing Theorem 6.2 and Theorem 5.3, we see that we can again get a reduction in variance depending on how small we can make { , −̂ , | −1 } 2 , i.e., how well we can predict the next response for each individuals at every time using past data. We end this section with an example demonstrating Theorem 5.3 and the consequent reduction in variance we can get using Theorem 6.2. Example 5 (Unit Varying Treatment Effects in Linear Models). Suppose that every individual has an engagement score that is a function of the previous day's engagement and covariate . Further suppose that each user has a user-specific treatment effect. More formally, , (1) = , (0) + , ∼ (20, 10 2 ) ,0 ∼ + ,0 , ∼ (25, 5 2 ), , ∼ Bern(0.5) for all , This scenario reflects a practical example where each unit may react differently to the treatment (even possibly having a negative treatment effect), but overall the treatment is effective in increasing the engagement level by 20 on average. For simplicity, we use a time-invariant covariate that also has a stationary relationship across time. We remark that we did not need to assume any of these as our method can account for any time varying effects or covariates. Lastly, all examples thus far have assumed the potential outcomes come from some independent distribution. Because the design-based approach conditions on the potential outcome, we can allow for any arbitrary dependence. In this case, we demonstrate it for an AR(1) process, where each user's current engagement is only dependent on previous engagement. We set = 20, = 100, = 1, = 0.5 for this example.
For the left panel of Figure 6, we build the confidence sequence using Theorem 5.3 by pretending we do not observe . For the right panel of Figure 6, we build the confidence sequence using Theorem 6.2, wherê , | −1 is the predicted response from an ordinary least square regression of all responses on all covariates available for time = 1, 2, … , . Consequently, as grows,̂ becomes more accurate, allowing a reduction in variance proportional to ( ) 2 . Figure 6 shows that using proxy outcomes substantially reduces the confidence sequence width. For example, the right panel would reject the null average treatment effect by = 6 while the left panel would reject it by = 43, approximately a seven times reduction.

Simulation study
Although all Examples 1-5 are technically simulations, they only demonstrate the properties for one confidence sequence, thus it is still unclear whether our proposed confidence sequences have the time-uniform  Table 1: The first two rows represent simulations under the same setting as that in Example 5. The third and fourth rows are also under the same setting except we change the potential outcome model to Equation (9). The last row shows the empirical type-1 error control when = 1 under scenario one. The second column represents the empirical proportion of times each respective confidence sequence covers the true treatment effect for all times when = 0.05. The third column represents the average time it took to reject the point null of zero treatment effect. The proportion and average are taken over 5000 Monte-Carlo simulated confidence sequences for each scenario.
type-1 error guarantee. In this section, we build upon Example 5 with three goals. First, we show the empirical uniform type-1 error guarantees obtained from our design-based asymptotic confidence sequences for all times (even early times). Second, we show the expected reduction in time to detect a statistically significant effect when incorporating proxy outcomes. Third, we further show the expected reduction in time to detect an effect when incorporating proxy outcomes from a misspecified prediction model. To achieve this, we first replicate the simulation in Example 5 for 5000 empirically computed confidence sequences and record the proportion of times it contained the true treatment effect for all times and the average time it took to reject the point null of zero treatment effect, which we refer to as "average stopping time." For the second scenario, we adjust the linear model with the following nonlinear model while using the same OLS prediction so that the prediction model is misspecified. Lastly, we also create a third scenario where we only have = 1 unit, i.e., a single time series experiment. The asymptotics are more credible in the panel data setting because even at = 1, there is effectively time steps already. Consequently, we should expect the panel data setting to have strong type-1 error guarantees even at early times. For this reason, we show the results when = 1 under the same linear model in Example 5, where we fix = 25, = 20 (we drop the subscript because there is only one unit) and the remaining scenario is identical. Although we could also use proxy outcomes for a single time series by again using the OLS estimator after ≥ 3 (so that the OLS estimator exists), we choose to not use the proxy outcome for brevity. We also only report the type-1 error and omit the average stopping time to facilitate comparison because the single time series with = 1 will have a substantially larger stopping time compared to the other panel data settings with = 20. Table 1 show the results of the simulation. As expected, scenarios 1-2 show strong type-1 error control even at early times. The over conservative coverage is likely due to our variance estimate being an upper bound estimate. Further, we see approximately a seven times reduction in the average stopping time using proxy outcomes under scenario 1. Although the difference is not as substantial when the prediction model is misspecified for a highly non-linear outcome, we still roughly see a 15% decrease in the average stopping time. Lastly, the third scenario shows that even when = 1, the time-uniform coverage guarantee holds for all time. Although the theory guarantees time-uniform coverage after a sufficiently large , our simulations suggest the coverage is strong even at early times likely because our estimated variance is conservative and the confidence width is large at early times. Nevertheless, for a single time series, we recommend practitioners to start "peeking" after some initial , e.g., ≥ 10. Incorporates covariates or any modeling assumption to reduce CS width Table 2: Summary of our contribution. The first column describes the confidence sequence of interest. The second column describes where to find the respective confidence sequence followed by key generalizations and remarks about the confidence sequence.

Concluding remarks: why design-based?
In this paper, we demonstrate how to build valid confidence sequences for various treatment effects across a variety of settings through a design-based approach. At this point, readers may wonder why the design-based approach allows such flexible and general results for sequential testing. We highlight two major reasons. First, the design-based approach allows assumption-light results because it avoids making any parametric, semi-parametric, or technical moment conditions about the potential outcomes. Since confidence sequences require technical constructions of martingales, which often rely on distributional assumptions, we can solely rely on the randomness induced by the treatment assignment to drive inference. In exchange, we require that the realizations of the potential outcomes are bounded, which is an innocuous assumption for most practical scenarios, and we further require that the treatment assignment probabilities are bounded away from zero or one. We acknowledge this assumption may be more restrictive as it may violate some practical scenarios where experimenters choose to deterministically assign treatment. Consequently, we view this as the main limitation of our approach.
Second, one may wonder if all the results are possible from a super-population view, where we assume the potential outcomes are from some distribution ( ) ∼ . In this case, if we make no further assumptions on how is varying across time, there is no way to have any consistent variance estimator for a single timeseries experiment since we only obtain . Lastly, in a dynamic setting with carryover effects, we would like to allow to depend on the past. With all these complications, it is unclear how to make progress in the super-population perspective.
Our work bridges the sequential testing literature with the design-based literature to perform continuous monitoring for the average treatment effect, contemporaneous treatment effect relevant also to the bandit settings, and the average contemporaneous treatment effect in panel data settings with time-uniform guarantees. We summarize our contribution and a few key remarks in Table 2. Our confidence sequences formally allow the analyst to "peek" at any time and stop the experiment in a data-dependent way, e.g., as soon as the confidence sequences do not cover zero while maintaining type-1 error guarantees through a design-based approach.

B Proof of Theorem 3.2
We build off the proof of Theorem 4 in (Howard et al., 2020). As hinted in Section 3.2, we will first show that is a non-negative supermartingale with respect to the filtration  , −1 . (Fan, Grama and Liu, 2015) show that exp + 2 ( + log(1 − )) ≤ 1 + for ≥ −1 and ∈ [0, 1). We let =̂ , where ≥ −1 since |̂ | ≤ for every by Assumption 2. Therefore, we have where the second line follows becausê 2 =̂ 2 and Lemma 2.1 and the last line follows because 1 − ≤ exp(− ). We plug = 1∕( + 1) and because the above is a non-negative quantity this directly implies that exp ∑ =1 (̂ − ) ( + 1) + 2 log + 1 + 1 + 1 is indeed a non-negative super martingale with respect to  −1 as desired with initial value less than one. We apply Lemma 3.1 and have that Taking =̃ ∕2 and applying the union bound completes the proof.

C Theoretical extension of Theorem 3.2
In this section, we correct the order of the confidence sequence presented in Theorem 3.2 for theoretical completeness. We leverage the results presented in (Waudby-Smith et al., 2022;Howard et al., 2020) by applying a mixture martingale over a truncated gamma distribution. The above proof shows that ∶= exp + ( + log(1 − )) is a super-martingale with initial value 1, where An upper confidence sequence can be obtained in a similar way. We remark that this confidence sequence does not solve the issue where it requires the analyst to know and before the experiment. Furthermore, this confidence sequence does not have a closed-form expression, thus it requires a root-solving algorithm to build the confidence sequence. However, (Waudby-Smith et al., 2022) show that this provably has an asymptotic rate of ( √ log( )∕ ), which does solve the issue related to the order of the confidence sequence width.

D Proof of Theorem 5.2
The proof proceeds in three steps. We note that this proof is based off the proof of Theorem 2.3 in (Waudby-Smith et al., 2021) but extended to our setting.
Step 1: Building martingale using Gaussian distribution Recently,  shows that all sequential tests must have an explicit or implicit construction of a non-negative martingale. Although one of the major advantages of an asymptotic confidence sequences is that it avoids explicitly constructing a martingale, the proof still relies on constructing a martingale with the asymptotic Gaussian distribution. Consequently, the first step of the proof builds a martingale from a sequence of standard Gaussian random variables.
By Lemma 5.1, { } is a martingale difference sequence with respect to , −1 . Similar to the proof of Step 2 of (Waudby-Smith et al., 2021), we also use the strong approximation theorem presented in (Strassen, 1967).
In particular, we require Equation (159) in Theorem 4.4 of Strassen's paper (further details in Lemma A.3 of (Waudby-Smith et al., 2021)) for our strong approximation theorem. However, our proof is different than that in Step 2 of (Waudby-Smith et al., 2021) for the following reason. The original Theorem 4.4 in (Strassen, 1967) is stated for martingales difference sequence of the form ( | ( 1 , … , −1 )) = 0, where we use Strassen's notation and are random variables with defined second moment. Although our martingale is of the form ( ( ) | ( 1 , … , −1 )) = 0, where (.) is the function that maps the data tô | −1 − ( 1∶( −1) ). More formally, to use the strong approximation theorem in (Strassen, 1967), we replace the beginning conditions of Theorem 4.4 in the following way.