{'fraction_non_alphanumeric': 0.04682481416903735, 'fraction_numerical': 0.01326053042121685, 'mean_word_length': 3.9521699613724155, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 28, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Synthetic Controls for Experimental Design

This article studies experimental design in settings where the experimental units are large aggregate entities (e.g., markets), and only one or a small number of units can be exposed to the treatment. In such settings, randomization of the treatment may result in treated and control groups with very different characteristics at baseline, inducing biases. We propose a variety of synthetic control designs (Abadie, Diamond and Hainmueller, 2010, Abadie and Gardeazabal, 2003) as experimental designs to select treated units in non-randomized experiments with large aggregate units, as well as the untreated units to be used as a control group. Average potential outcomes are estimated as weighted averages of treated units, for potential outcomes with treatment -- and control units, for potential outcomes without treatment. We analyze the properties of estimators based on synthetic control designs and propose new inferential techniques. We show that in experimental settings with aggregate units, synthetic control designs can substantially reduce estimation biases in comparison to randomization of the treatment.


Introduction
Consider the problem of a ride-sharing company choosing between two compensation plans for drivers (Doudchenko et al., n.d.;Jones and Barrows, 2019). The company can either keep the current compensation plan or adopt a new one with higher incentives. In order to estimate the effect of a change in compensation plans on profits, the company's data science unit designs an experimental evaluation where the new plan is deployed at a small scale, say, in one of the local markets (cities) in the country. In this setting, a randomized control trial -or A/B test, where drivers in a local market are randomized into the new plan (active treatment arm) or the status-quo (control treatment arm) -is problematic. On the one hand, such an experiment raises equity concerns, as drivers in the same local market but in different treatment arms obtain different compensations for the same jobs. On the other hand, if drivers in the active treatment arm respond to higher incentives by working longer hours, they will effectively steal business from drivers in the control arm of the experiment, which will result in biased experimental estimates. 1 A possible approach to this problem is to assign an entire local market to treatment, and use the rest of the local markets, which remain under the current compensation plan during the experimental periods, as potential comparison units. In this setting, using randomization to assign the active treatment allows ex-ante (i.e., pre-randomization) unbiased estimation of the effect of the active treatment. However, ex-post (i.e., post-randomization) biases can be large if, at baseline, the treated unit is different from the untreated units in the values of the features that affect the outcomes of interest.
As in the ride-sharing example where there is only one treated local market, large biases may arise more generally in randomized studies when either the treatment arm or the control arm contains a small number of units, so randomized treatment assignment may not produce treated and control groups that are similar in their features. In those cases, the fact that estimation biases would have averaged out over alternative treatment assignments is of little comfort to a researcher who is, in practice, limited to one assignment only.
To address these challenges, we propose the use of the synthetic control method (Abadie, Diamond andHainmueller, 2010, Abadie andGardeazabal, 2003) as an experimental design to select treated units in non-randomized experiments, as well as the untreated units to be used as a comparison group. We adopt the name synthetic control designs to refer to the resulting experimental designs. 2,3 1 A randomized evaluation across many markets is a potential solution to the problem of experimental interference between drivers. In practice, however, large-scale market-level randomized evaluations are often unfeasible. In the context of the ridesharing company example, large-scale market-level randomized evaluations (i) could be prohibitively expensive, (ii) could still raise substantial equity concerns, (iii) could create great disappointment among the large number of drivers in the treated cities if the program is rolled back after experimentation, and (iv) in some cases, the number of cities where the company operates could be too small for effective randomization.
2 While we leave the "experimental" qualifier implicit in "synthetic control design", it should be noted that the synthetic control designs proposed in this article differ from observational synthetic control designs (e.g., Abadie, Diamond and Hainmueller, 2010, Abadie and Gardeazabal, 2003, Doudchenko and Imbens, 2016, for which the identity of the treated unit(s) is taken as given.
3 See, e.g., Abadie (2021), Amjad, Shah and Shen (2018), Arkhangelsky et al. (2021), Doudchenko and Imbens In our framework, the choice of the treated unit (or treated units, if multiple treated units are desired) aims to accomplish two goals. First, it is often useful to select the treated units such that their features are representative of the features of an aggregate of interest, like an entire country market. The treatment effect for the treated units selected in this way may more accurately reflect the effect of the treatment on the aggregate of interest. Second, the treated units should not be idiosyncratic in the sense that their features cannot be closely approximated by the untreated units. Otherwise, the reliability of the estimate of the effect on the treated unit may be questionable. We show how to achieve these two objectives, whenever they are possible to achieve, using synthetic control methods.
While we are aware of the extensive use of synthetic control methods for experimental design in data science units, especially in the technology industry, 4 the academic literature on this subject is at a nascent stage. There are, however, a few publicly available studies that are connected to this article. Aside to the present article, to our knowledge, Doudchenko et al. (n.d.) and the relevant population. The inferential methods in the present article also differ from those in the related literature. In particular, Doudchenko et al. (2021) propose a permutation procedure for inference that requires that potential outcomes without the treatment are independent and identically distributed (i.i.d.) in time. In contrast, the inferential procedure proposed in the present article allows for time series dependence and non-stationarity in outcomes, which are pervasive features of time-series data. Another important difference between the present article and Doudchenko et al. (n.d.) and Doudchenko et al. (2021) is that Doudchenko et al. (n.d.) and Doudchenko et al. (2021) make use of pre-treatment outcomes only to select treated and control units, while our method allows the use of other observed features of the units. Agarwal, Shah and Shen (2021) propose synthetic interventions, a framework related to synthetic controls, and apply it to estimate treatment effect heterogeneity in an experimental setting with multiple treatments. Their work primarily focuses on the analyses of experimental data, but not on the design of experiments. Bottmer et al. (2021) is also related to the present article in the sense that they study synthetic control estimation in an experimental setting. Their article, however, considers only the case when the treatment is randomized, and is not concerned with issues of experimental design.

Synthetic Control Designs
We consider a setting with T time periods and J units, which may represent J local markets as in the ride-sharing example in the previous section. Let T 0 be the number of pre-experimental periods, with 1 ≤ T 0 < T . At the end of period T 0 , a researcher designs an experiment to conduct during periods T 0 + 1, T 0 + 2, . . . , T . Using information available at T 0 , the experimenter aims to select the set of units that will be administered treatment (intervention) during the experimental periods.
To define causal parameters, we formally adopt a potential outcomes framework. For any j ∈ {1, . . . , J} and any t ∈ {T 0 + 1, . . . , T }, let Y I jt be the potential outcome for unit j at time t when the unit is exposed to treatment starting at T 0 + 1. Similarly, for any j ∈ {1, . . . , J} and any t ∈ {1, . . . , T }, let Y N jt be the potential outcome for unit j at time t under no treatment. We assume that the outcome variable of interest is scaled so that it does not depend on the size of the unit. In the ridesharing example, Y I jt and Y N jt could measure net income divided by market size, under the active and the control treatment, respectively. Unit-level treatment effects are defined as Among units with w j = 0, those with v j > 0 are used to estimate average outcomes under no intervention.
The first goal of the experimenter is to choose w 1 , . . . , w J such that for t = T 0 + 1, . . . , T . If equation (1) holds, a weighted average of outcomes for the units selected for treatment reproduces the average outcome with treatment for the entire population of J units.
In practice, however, the choice of w 1 , . . . , w J cannot directly rely on matching the population average of Y I jt , as in equation (1). The quantities Y I jt are unobserved before time T 0 + 1, and will remain unobserved in the experimental periods for the units that are not exposed to the treatment. Instead, we aim to approximate equation (1) using predictors observed at T 0 of the values of Y I jT 0 +1 , . . . , Y I jT . Note also that it is not possible to use the weights w 1 = f 1 , . . . , w J = f J , because it would leave no units in the donor pool, making the set of units with v j > 0 empty and violating the second condition in (2).
The second goal of the experimenter is to choose v 1 , . . . , v J such that or, alternatively, If equations (2) or (3) hold, a weighted average of outcomes for the units in the donor pool reproduces the average outcome without treatment for the entire population of J units (equation (2)), or for the units selected for treatment (equation (3)). Like in the previous case with treated outcomes, it is not feasible to directly choose v 1 , . . . , v J so that equation (2) or (3) is satisfied.
Instead, we propose a variety of methods to approximate either (2) or (3) based on predictors of Y N jT 0 +1 , . . . , Y N jT . For the treated units, we define Y jt = Y N jt if t = 1, . . . , T 0 , and Y jt = Y I jt if t = T 0 + 1, . . . , T . For the untreated units, we define Y jt = Y N jt , for all t = 1, . . . , T . That is, Y jt is the outcome et al., 2021).
We choose w = (w 1 , . . . , w J ) and v = (v 1 , . . . , v J ) to match the pre-intervention values of predictors of the potential outcomes Y N jt and Y I jt for t > T 0 . Let X j be a column vector of pre-intervention features of unit j. We see the features in X j as predictors of the values of Y N jt and Y I jt in the post-intervention periods, in a sense that will be made precise in Section 3. We use the notation That is, X is the vector of population values for the predictors in X j . For any real vector x, x is the Euclidean norm of x, and x 0 is the number of non-zero coordinates of x. Let m and m be positive integers such that 1 ≤ m ≤ m ≤ J − 1. A simple selector of w = (w 1 , . . . , w J ) The first term of the objective function in (4) measures the discrepancies between the population average of the features in X j (f -weighted) and the averages of the features for units assigned to the treatment group (w-weighted). The second term is analogous but with the second average taken over the units assigned to no intervention (v-weighted). The first four constraints require that the weights in w, as well as the weights in v, are non-negative and sum to one. They also require that any unit selected for treatment cannot be utilized as a control unit -so, if w j > 0, then v j = 0. The last constraint allows a minimum and maximum number of units assigned to treatment. This restriction is of practical importance in a variety of contexts, especially when experimentation is costly and the experimenter is restricted in the number of units that may where the j-th coordinate of c is equal to the cost of assigning unit j to treatment, d is a Jdimensional vector with ones for coordinates with w j > 0, and zeros otherwise, and B is the experimenter's budget.
Let w * = (w * 1 , . . . , w * J ) and v * = (v * 1 , . . . , v * J ) be a solution to the optimization problem in (4). In practice, we do not require optimality of (w * , v * ), as long as (w * , v * ) is feasible and where 0 is a vector of zeros of the same dimension as X j . Suppose that units with w * j > 0 are assigned to treatment in the experiment, and units with w * j = 0 are kept untreated. A synthetic control estimator of τ t is τ t = τ t (w * , v * ), i.e., This estimator is based on approximations to equations (1) and (2) that rely on X j , the observed predictors of the potential outcomes, Y N jt , and Y I jt . Note that for every solution to (4) with m ≤ v 0 ≤ m, there is another solution that swaps the roles of the treated and the untreated in the experiment without changing the objective value.
In what follows, we take the weight selector in (4) as a starting point for synthetic control designs, and modify it in several respects. A second formulation of the synthetic control design is based on equations (1) and (3), The parameter β > 0 reflects the trade-off between selecting treated units to fit the aggregate value of the predictors X, and selecting control units to fit the aggregate value of the treated units. A small value of β favors designs with treated units that closely match X. A large value of β, instead, favors designs with aggregate treated and aggregate control units that closely match each other. While it is possible to use data-driven selectors of β, the rule of thumb β = 1 provides a natural choice, which equally weights the two terms in the objective function in (6).
For this formulation of the synthetic control design, treatment assignment and estimation follow the same procedures as with the previous one. Large values for β produce estimators that target the w-weighted average effect of the treatment on the treated, τ T t of Doudchenko et al. (2021). Small values of β prioritize estimation of the average treatment effect, τ t .
In our third formulation of the synthetic control design, the experimenter selects a synthetic treated unit to match the average values of the characteristics in the population. However, unlike in the design in (6), the experimenter chooses multiple synthetic controls, one for each unit that contributes to the synthetic treated unit. For any J-dimensional vector of non-negative The parameter ξ > 0 arbitrates potential trade-offs between selecting treated units to fit the Let {w * j , v * ij } i,j=1,...,J be a solution of the optimization problem in (7). As before, we do not strictly require optimality of Assign units with w * j > 0 to treatment in the experiment, and keep units with Then, Note: Panels (a) and (b) plot the values of the predictors in X j , which is bivariate in this simple example. Units assigned to treatment are drawn in red. In panel (a) we treat the entire sample as a single cluster. In panel (b) we divide the sample into three clusters and assign one unit in each cluster to the treatment.
Our next adjustment to the synthetic control design is motivated by settings where the units may be naturally divided in clusters with similar values in the predictors, X 1 , . . . , X J . For example, weather patterns, which may be highly dependent across cities in the same region (e.g., Northeast, Midwest, etc, in the US), may influence the seasonality of the demand for ridesharing services. In those cases, it is natural to treat each cluster (each region, in our example) as a distinct experimental design to ameliorate interpolation biases. Figure 1  They closely reproduce the value of X, but they all fall in the same central cluster, far away from observations in other clusters. In panel (b), assignment to treatment takes into account the clustered nature of the data, and one unit is treated per cluster. This provides a better approximation of the distribution of the predictor values for the entire sample, ameliorating concerns of interpolation biases.
Suppose we divide the set of J available units into K clusters. Let I k be the set of indices for the units in cluster k. The cluster mean is for each cluster k = 1, . . . , K. For each index i = 1, . . . , J, let k(i) be the cluster that unit i belongs to, i.e., i ∈ I k(i) . A clustered version of the synthetic control design in (7) is given by: min w j ,∀j=1,2,..,J, v ij ,∀i,j=1,2,...,J We conclude this section by discussing other possible extensions to the synthetic control design. First, it is well known that synthetic control estimators may not be unique. Lack of uniqueness is typical in settings where the values of the predictors that a synthetic control is targeting (i.e., X in equation (4), or X j for a treated unit in equation (7) Here, λ 1 and λ 2 are positive constants that penalize discrepancies between the target values of the predictor X and the values of the predictors for the units that contribute to their synthetic counterparts. 5 Other types of penalization are possible. In particular, Doudchenko and Imbens (2016) Appendix A provides details on how to apply bias correction techniques in a synthetic control design.

Formal Results
We first introduce an extension of the linear factor model commonly employed in the synthetic control literature, which we use to analyze the properties of estimators that are based on a synthetic control design.
Assumption 1 Potential outcomes follow a linear factor model, where Z j is a (r × 1) vector of observed covariates, θ t and γ t are (r × 1) vectors of unknown parameters, µ j is a (F × 1) vector of unobserved covariates, λ t and η t are (F × 1) vectors of unknown parameters, and jt and ξ jt are unobserved random shocks.
Equation (10a) is the linear factor model for potential outcomes under no treatment that is commonly employed in the literature as a benchmark model to analyze the properties of synthetic control estimators (see, e.g., Abadie, Diamond andHainmueller, 2010, Ferman, 2020). Equation (10b) extends the linear factor structure to potential outcomes under treatment. The reason for extending the linear factor model to treated outcomes is that, in contrast to the usual setting of synthetic control estimation with observational data, experimental synthetic control designs require the choice of a treatment group in addition to the choice of a comparison group.
We employ the covariates in Z j as well as pre-experimental values of the outcome variables Y jt to construct the vectors of predictors, X j . In particular, let E ⊆ {1, . . . , T 0 }, T E = |E|, and let Y E j be the (T E × 1) vector of T E pre-intervention outcomes for unit j and time indices in E. We define for j = 1, . . . , J. That is, the vector of predictors, X j , collects the covariates in Z j and the pre-intervention outcome values, Y jt , for the "fitting periods" in E. In practice, the values in X j are often scaled to make them independent of units of measurement, or to reflect the relative importance of each of the predictors (see, e.g., Abadie, 2021).
The next assumption gathers regularity conditions on model primitives.

Assumption 2
(i) Let λ t,f and η t,f be the f -th coordinates of λ t and η t , respectively. There existsλ andη, Unless otherwise noted, all probability statements are over the joint distribution of jt and ξ jt and conditional on the values of the other components on the right-hand sides of equations (10a) and (10b). The next assumption pertains to the quality of the synthetic control fit. For concreteness, we focus on the base design in (4), and choose where w * = (w * 1 , . . . , w * J ) and Assumption 3 implies that the synthetic treated and control units defined by w * and v * provide a perfect fit for X. Assumption 3 is a strong restricition, which may only hold approximately in practice.
The bias bound in Theorem 1 depends on the ratio between the scale of jt , represented in (12) by σ, and the number of fitting periods T E . Intuitively, the bias of the synthetic control estimator is small when a good fit in pre-intervention outcomes (Assumption 3) is obtained by implicitly fitting the values of the latent variables, µ j . Overfitting happens when pre-intervention outcomes are instead fitted out of the variability in the individual transitory shocks, jt . A small number of fitting periods, T E , combined with enough variability in jt increases the risk of overfitting and, as a result, increases the bias bound. Similarly, for any fixed value of T E , the bias bound increases with J, reflecting the increased risk of over-fitting created by increased variability in jt over larger donor pools. Finally, the number of unobserved factors, F , enters the bound linearly, which highlights the importance of including the observed predictors, Z j -other than pre-intervention outcomes -in the vector of fitting variables, X j . Under the factor model in equations (10a) and (10b), observed predictors not included in Z j are shifted to µ j , increasing F and the magnitude of the bias bound.
We next turn our attention to inference. We utilize a set of "blank periods," B ⊆ {1, . . . , T 0 }\ E, which comprise pre-intervention periods whose outcomes, Y jt , have not been used to calculate w * or v * . Because pre-intervention periods that are not in E or B could always be discarded from the data, we can consider B = {1, . . . , T 0 } \ E only, without loss of generality. We therefore assume that the number of elements of B is T B = |B| = T 0 − T E . We aim to test the null: For t = T 0 + 1, . . . , T , and j = 1, . . . , J, where ξ jt has the same distribution as jt .
Under the null hypothesis in (13), the distribution of Y I jt is the same as the distribution of Y N jt , for t = T 0 + 1, . . . , T , and j = 1, . . . , J (but the realized values of Y I jt and Y N jt may differ). Recall from (5) that, for t ∈ {T 0 + 1, . . . , T }, a synthetic control estimator is defined as Let u t = τ t , ∀t ∈ {T 0 + 1, . . . , T } be the synthetic control estimator on the post-intervention periods. Similarly, for each t ∈ B in the blank periods, let Such u t for t ∈ B are "placebo" treatment effects estimated for the blank periods. We study the properties of a test based on combinations from the set { u t : t ∈ B ∪ {T 0 + 1, . . . , T }}.
In addition, let e = ( u T 0 +1 , . . . , u T ) = ( τ T 0 +1 , . . . , τ T ). This is a vector of treatment effect estimates from the post-intervention periods. For any (T −T 0 )-dimensional vector e = (e 1 , . . . , e T −T 0 ), we adopt the test statistic, Other choices of test statistics are possible, such as those based on an L p -norm of e (Chernozhukov, Wüthrich and Zhu, 2021) and one-sided versions of the resulting test statistics (i.e., with e t or −e t replacing |e t | in equation (14)). Let be the p-value based on (14). Theorem 2 below shows that if λ t are exchangeable random variables for t ∈ B ∪ {T 0 + 1, . . . , T }, then a test of the null hypothesis in (13) based on the p-value in (15) is exact.
Theorem 2 Suppose that Assumptions 1-3 hold, that for j = 1, . . . , J, t = 1, . . . , T , jt are continuously distributed, and that for t ∈ B ∪ {T 0 + 1, ..., T } the unknown parameters λ t are exchangeable random variables. Then, Note that, under the assumptions of Theorem 2, the potential outcome series Y N jt is allowed to be non-stationary through the term δ t + θ t Z j in equation (10a). This is in contrast with a related result in Doudchenko et al. (2021), which requires that the potential outcomes Y N jt are i.i.d. in time. Exchangeability of λ t is a strong restriction. Theorem A.1 in Appendix A relaxes this restriction by showing that conditional on λ t (i.e., without resorting to exchangeability of λ t ), the p-value in (15) is still approximately valid for large T E .
In some settings, the number of possible combinations, |Π|, could be very large, making exact calculation of p computationally expensive. In those instances, random samples of the combinations in Π can be used to approximate the p-value in equation (15).
The inferential technique proposed in this article is related to, but distinct from, the permutation methods in Abadie, Diamond and Hainmueller (2010) require both a novel testing procedure that takes advantage of the availability of blank periods and proof techniques that are, to our knowledge, new to the literature.

Base Results
We report in this section the results of a simulation study that illustrates the behavior of the estimators proposed in this article. We consider a setting with J = 15 units, r = 7 observable covariates and F = 11 unobservable covariates. We simulate data for T = 30 periods in total, with T 0 = 25 pre-intervention periods, and T − T 0 = 5 experimental or post-intervention periods.
We use the factor model in Assumption 1 to generate potential outcomes. For t = 1, . . . , T , we generate the series δ t and υ t as small-to-large re-arrangements of T i.i.d. Uniform (0, 20) random variables. For j = 1, . . . , J, we set both Z j and µ j to be random vectors of i.i.d. Uniform (0, 1) random variables. For t = 1, . . . , T , we set θ t , γ t , λ t , and η t to be random vectors of i.i.d. Uniform (0, 10) random variables. Finally, for j = 1, . . . , J, and any t = 1, . . . , T , we set jt and ξ jt to be i.i.d. Normal (0, σ 2 ) random variables, with σ 2 = 1. In Online Appendix D we present additional simulation results of alternative values of the noise parameter σ 2 .
Using the data generating process described above, we draw a single sample and conduct the synthetic control design in (4) . . , T . The black dashed line represents the trajectory of the synthetic control unit J j=1 v * j Y jt , for t = 1, . . . , T . The synthetic treated and synthetic control units closely track each other in the pre-experimental periods. They diverge in the experimental periods, when a treatment effect emerges as a result of the differences in the parameters of the data generating processes for Y N jt and Y I jt . Figure 3 reports the difference between the synthetic treated and the synthetic control outcomes. The inferential procedure of Section 3 produces p-value equal to 0.0040 for the null hypothesis of no treatment effect in (13).

A Comparison of Synthetic Control Designs
In this section we compare the performance of different synthetic control designs under the same data generating process as in Section 4.1. We report the estimated average treatment effects and the synthetic treated and synthetic control weights for a single simulation. In Section 4.3, we compare the performance of different synthetic control designs over multiple simulations.
We consider five varieties of the synthetic control design: 1. Unconstrained design: This is the design in (4) without a cardinality constraint, so m = 1 and m = J − 1 = 14.
4. Unit-level design: This is the design in (7), which fits a different synthetic control to each unit assigned to treatment. We vary ξ from 0.01 to 100.
Section C in the Online Appendix discusses computational issues.

Average Treatment Effects
The first panel of Table 1 reports the average treatment effects, τ t . The first five columns in the second panel report average treatment effect estimates, τ t , for periods T 0 + 1 = 26 through T = 30, under different synthetic control designs. The last three columns in the second panel report the mean absolute error (MAE) and the mean squared error (MSE), defined as as well as the p-value in (15).
Because Table 1 reports outcomes for a single simulation only, the results may not be reflec- tive of general patterns over many simulations, even for the particular data generating process employed to produce the data. However, we use a single simulation to illustrate patterns in the estimates that are induced directly by the features of their respective estimators. In Section 4.3 we report averages over a large number of simulations.
The Unconstrained estimates in Table 1

Synthetic Treated and Synthetic Control Weights
Tables 2 and 3 report the synthetic treated and synthetic control weights (w * and v * , respectively) for the designs of Table 1. For the Unit-level design, synthetic control weights are aggregated as in (8). For the Unconstrained and Penalized designs, the synthetic treated and synthetic control weights can always be swapped without changing the objective values for their respective designs. For the Constrained design, the weights can be swapped when v * 0 ≤ m. When it is possible to swap synthetic treated and synthetic control weights, we choose the treated units so that the number of units with positive weights in w * is smaller than the number of units with positive weights in v * . When w * 0 = v * 0 , we determine whether or not to swap following a specific rule described in Section C in the Appendix.
The Constrained design imposes sparsity in the synthetic treatment weights through a hard cardinality constraint specified by the integer m. For m = 7, the Constrained and the Unconstrained weights coincide. For the Unit-level design, large values of ξ generate sparsity in the synthetic treated weights. A sufficiently large value of ξ produces a Unit-level design where the only treated unit is the unit that can be most closely fitted by a convex combination of the other ones. For large values of λ, the Penalized design behaves like a one-to-one matching design, assigning all the weight to one treated and one control unit. For small values of λ, the Penalized design weights are close to the Unconstrained design weights.

Performance Over Multiple Simulations
In this section, we compare the five synthetic control designs of Section 4.2 across averages over 1000 simulations that independently generate the model primitives (i.e., the factor loadings, covariates, and error terms) of Assumption 1. The data generating process is the same as in Section 4.1.

Average Treatment Effects
The first panel of Table 4 reports average treatment effects, τ t , averaged over 1000 simulations.
The second panel reports estimated average treatment effects, mean absolute error, mean squared error, and p-value, all averaged over simulations, as well as rejection rates. Because the treatment     effect is different from zero in the simulation of Table 4, smaller p-values and larger rejection rates reflect better performance of the testing procedure for a particular design.
In Table 4, the Unconstrained design has a strong relative performance. The performance of the Constrained design improves for larger m, and is virtually identical to the performance of the

Average Treatment Effects on the Treated
The first five columns in Table 5 report averages of τ T t , the average effects of treatment on the treated units. These quantities depend on the weights for the treated units, which are different across different formulations of the synthetic control design. The next five columns report averages of τ t . They are the same as in Table 4, yet we use them as estimators for τ T t in Table 5. The last two columns of Table 5 report the mean absolute error and the mean squared error, defined as in (16) but with τ T t replacing τ t . The results in Table 5 are qualitatively similar to those for τ t in Table 4, with one notable exception. As expected, for intermediate and large values of β, the Weakly-targeted design outperforms the other designs when the goal is to estimate τ T t . This is because in the Weakly-targeted design the synthetic control weights are targeted to τ T t (and more so as β becomes large).

Test size
In this section, we generate the model primitives under the null hypothesis (13). That is, we employ a data generating process such that the values of the common factors and the distributions of the idiosyncratic error variables are unaffected by the intervention.
We report the simulation results in Table 6, which organizes information in the same way as in Table 4. Because the data are generated from the same distribution under treatment and under no treatment, the average treatment effects in Table 6 are close to zero. The same is true for the averages of τ t for all designs. Under the null hypothesis (13), the p-value should approximately follow a uniform distribution between zero and one. The results in Table 6 show good behavior of our testing procedure under the null hypothesis: average p-values and rejection rates are close to 0.5 and 0.05, respectively.

Performance with Nonlinearities
In this section, we study the behavior of the estimators based on synthetic control designs under deviations from the linear model in (10a) and (10b). We consider a nonlinear data generating process, The reason to study a nonlinear model is that nonlinearities may induce interpolation biases, affecting the relative performance of the different designs. All parameter values are the same as in the simulation setup of section 4.1, except for the values of θ t , γ t , λ t , and η t , which are chosen to be random vectors of i.i.d. Uniform (0, 3) random variables, instead of Uniform (0, 10), to control the magnitude of the exponential components in the nonlinear design.
Tables 7 and 8 report the results for τ t and τ T t , respectively. In comparison to the results in Tables 4 and 5, we now see that the Unit-level and Penalized designs can easily match and in some cases improve the performance of the Unconstrained design, especially for the estimation of τ T t . By fitting each treated unit with a unit-specific synthetic control, the Unit-level design can ameliorate interpolation biases. The Penalized design selects synthetic treated and control units close to X in the space of the predictors, which can reduce interpolation biases at the potential cost of lower precision for large values of λ (in which case, the Penalized design employs a small number of units in the synthetic treated and synthetic control). As in Table 8, when the parameter of interest is τ T t , the weakly targeted design easily outperforms the unconstrained estimator for large values of β.

Comparison to Randomized Control Trials
Using randomization to assign treatment allows ex-ante (pre-randomization) unbiased estimation of the average treatment effect. However, as we show below, ex-post (post-randomization) biases can be large, especially when only a small number of units are treated. Only the first term on the right-hand side of (B.4) has a non-zero mean (because the weights, w * j , depend on the error terms E j ). Therefore, Using the same line of reasoning for the second term on the right-hand side of (B.1), we obtain For any t ≥ T 0 + 1 and s ∈ T E , under Assumption 2 (ii), we apply Cauchy-Schwarz inequality and the eigenvalue bound on the Rayleigh quotient to obtain Similarly, Because¯ E jt is a linear combination of independent sub-Gaussians with variance proxy σ 2 , it follows that¯ E jt is sub-Gaussian with variance proxy (ηλF/ζ) 2 σ 2 /T E . Let q ≥ 1 be any positive integer. We obtain The first and third inequalities are implied by Jensen's Inequality, the second inequality is implied by Holder's inequality, and the last inequality is bounding the absolute moments of sub-Gaussian random variables (see, e.g., Rigollet and Hütter, 2019, Lemma 1.4). An analogous argument yields, It follows that for t ∈ B ∪ {T 0 + 1, . . . , T }. Since for t ∈ B ∪ {T 0 + 1, ..., T }, the u t are exchangeable, and λ t are exchangeable for t ∈ B ∪ {T 0 + 1, . . . , T } and independent of jt , then u t are exchangeable for t ∈ B∪{T 0 +1, . . . , T }, and the result of the theorem follows from Theorem D.1 in Chernozhukov, Wüthrich and Zhu (2021).

C. Swapping Treated and Control Weights
Recall that when it is possible to swap synthetic treated and synthetic control weights, we choose the treated units so that the number of units with positive weights in w * is smaller than the number of units with positive weights in v * . When w * 0 = v * 0 , we determine whether or not to swap using the following rule. For the Unconstrained design, we choose the treated group to be the one with the smallest index among the units with positive weights. In particular, in the Unconstrained design in Tables 2 and 3, the unit in the synthetic treated with lowest index is the 1 st unit, and the unit in the synthetic control with the lowest index is the 4 th unit. We use the same procedure based on lowest index for Constrained with m = 7 (highest value) and Penalized with λ = 0.01 (lowest value). Then, starting from m = 7 and for smaller values of m, we assign to the treated group the set of weights that is most similar to the weights obtained for w * 0 ≤ m + 1 (in terms of what units obtain positive weights). In those cases where the two sets of swappable weights for w * 0 ≤ m are equally similar to the synthetic treated weights for w * 0 ≤ m + 1, we select the set of weights with the smallest index. We follow the analogous procedure for λ > 0.01, starting from smaller values of λ.
Next, define the following two functions. Let The proof of Theorem A.1 proceeds in four steps. In step one, we define a high probability event, C 1 , such that u t and u t are close to each other under C 1 . In step two, we define a high probability event, C 2 , such that many components of {S(e π )} π∈Π are well-separated from S(e) under C 2 .
In step three, we show that, conditional on C 1 and C 2 , the ordering of S(e π ) and S(e) will be the same as the ordering of S( e π ) and S( e) for most π ∈ Π, which implies that F (S( e)) and F (S(e)) are also close to each other. In step four, we conclude the proof by linking F (S( e)) to the estimated p-value, andF (S(e)) to the nominal level α.  (2014)) Let X 1 , X 2 , . . . , X n be independent and continuously distributed random variables with densities f X 1 , f X 2 , . . . , f Xn . For any k ∈ {1, 2, . . . , n}, let Λ X k be the smallest upper bound on the probability density f X k . Suppose that for any k ∈ {1, 2, . . . , n}, Λ X k ≤ κ. Then for any a 1 , a 2 , . . . , a n such that n k=1 a 2 k = 1, Λ (a 1 X 1 +a 2 X 2 +...+anXn) ≤ √ eκ.
Lemma B.3 Let X be a continuously distributed random variable with a density f X . Let Λ X be the smallest upper bound on the probability density f X .
1. The random variable |X| has a density f |X| bounded by Λ |X| ≤ 2Λ X ; 2. For any constant a = 0, the random variable aX has a density, f aX , bounded by Λ aX ≤ Λ X /a; Proof of Lemma B.3. To prove 1, note that for any v ≥ 0, To prove 2, note that for any v ≥ 0, for t ∈ B ∪ {T 0 + 1, . . . , T }. We next define an event Note that, The above indicator involves a total of 2k many |u t |'s. Intuitively, it is obtained by canceling out common terms in S(e π ) and S(e).
Below we focus on the properties of the sum of such indicators. First, focus on the probability density of t∈π\π 0 |u t | − t∈π 0 \π |u t | . We have where the first inequality is due to Lemma B.3-1; the second inequality is due to Lemma B.2-2; the third inequality is due to Lemma B.2; the last inequality is due to Lemma B.4 and B.3-1.

C. Implementations of the Optimization Formulations
To computationally solve (4), i.e., the Unconstrained design, we propose two methods. The first method is by enumeration, which takes advantage of the objective function of (4) being separated between w and v. If we knew which units were to receive treatment and which units were to receive control, then we could decompose (4) into two classical synthetic control problems and solve both of them efficiently. We brute force enumerate all the possible combinations of the treatment units and control units. Because the two groups of treated and control units can be swapped (see Section 4.2.2), we only enumerate combinations such that the cardinality of the treated group is smaller than or equal to the cardinality of the control group. When the cardinality of the treated group is equal to the cardinality of the control group, we prioritize the treated group to be the one with the smallest index among the units with positive weights.
The second method solves a constrained optimization problem, by converting it into the canonical form of a Quadratic Constraint Quadratic Program (QCQP), which we detail below. The decision variables are w j and v j , ∀ j = 1, . . . , J. For simplicity, we write it in a vector formW = (w 1 , w 2 , ..., w J , v 1 , v 2 , ..., v J ).
Let M be the dimension of the predictors X j . Let X be an M × J matrix, each column of which is X j , which stands for the predictors of unit j.
Using the above notations we re-write the (non-convex) QCQP as follows, minW P 0W + q 0 W (C.1) s.t. e 1W = 1, e 2W = 1, The first computational method (enumeration) solves two synthetic control problems in each iteration. The synthetic control problems can be efficiently solved. We implement the synthetic control problem using the "lsei" function from "limSolve" package in R 4.0.2. For the second computational method (quadratic programming), the problem (C.1) is implemented using Gurobi 9.0.2 in R 4.0.2. Since the QCQP is non-convex, the computation leads to some numerical errors up to 0.001 in finding the treated and control weights. So we round the treated and control weights to the nearest 2-digits in the implementation of the QCQP. Moreover, for all the weights that are less than or equal to 0.01, we trim the weights to zero. This is because smaller weights suffer from greater impacts of numerical errors, and that numerical errors could make zero weights to be non-zero, thus having a non-negligible impact on the swapping rule.
To conclude, we compare the treated and control weights calculated from both methods. Both methods yield the same treated and control weights up to some negligible rounding error, while the first method takes longer computational time.
The other designs are computationally implemented using either one of the above two methods.
The Constrained design is implemented using the enumeration method. In cases when the cardinality constraint m is small, this brute force enumeration is very efficient. The Weakly-targeted design is implemented using the quadratic programming method. In the QCQP formulation, the objective function has both a different quadratic term P 0 and a different linear term q 0 .
The Unit-Level design is implemented using the enumeration method. The Penalized design is implemented using the quadratic programming method. In the QCQP formulation, the objective function has the same quadratic term P 0 and a different linear term q 0 .

D. Additional Simulation Results
In Section 4.1 the idiosyncratic shocks are i.i.d. Normal with variance σ 2 = 1. Figures 4 and 5 report results for σ 2 = 5 and σ 2 = 10, respectively. Figures 6 and 7 report differences between the outcomes for the synthetic treated and the synthetic control units for the same values for σ 2 . As the value of σ 2 increases, the quality of the post-treatment estimation and inference deteriorates, and the p-value for the null hypotheses of in (13)