{'fraction_non_alphanumeric': 0.034521635764267596, 'fraction_numerical': 0.017352728520483817, 'mean_word_length': 4.650716709369157, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Augmented Synthetic Control Method

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in panel data settings. The"synthetic control"is a weighted average of control units that balances the treated unit's pre-treatment outcomes as closely as possible. The curse of dimensionality, however, means that SCM does not generally achieve exact balance, which can bias the SCM estimate. We propose an extension, Augmented SCM, which uses an outcome model to estimate the bias due to covariate imbalance and then de-biases the original SCM estimate, analogous to bias correction for inexact matching. We motivate this approach by showing that SCM is a (regularized) inverse propensity score weighting estimator, with pre-treatment outcomes as covariates and a ridge penalty on the propensity score coefficients. We give theoretical guarantees for specific cases and propose a new inference procedure. We demonstrate gains from Augmented SCM with extensive simulation studies and apply this framework to canonical SCM examples. We implement the proposed method in the new augsynth R package.


Introduction
The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit in settings with a modest number of control units and with many pre-treatment outcomes for all units (Abadie and Gardeazabal, 2003;Abadie et al., 2010Abadie et al., , 2015. The idea is to construct a weighted average of control units, known as a synthetic control, that matches the treated unit's pre-treatment outcomes. The estimated impact is then the difference in post-treatment outcomes between the treated unit and the synthetic control. SCM has been widely applied -the main SCM papers have over 4,000 citations -and has been called "arguably the most important innovation in the policy evaluation literature in the last 15 years" ).
An important limitation of this approach is that, while SCM minimizes imbalance in pretreatment outcomes, it generally fails to achieve exact balance due to the curse of dimensionality (Ferman and Pinto, 2018). The resulting imbalance can lead to bias.
To address this, we propose the augmented synthetic control method (ASCM). Analogous to bias correction for inexact matching (Abadie and Imbens, 2011), ASCM uses an outcome model to estimate the bias due to covariate imbalance and then de-biases the original SCM estimate. If the estimated bias is small, then the SCM and ASCM estimates will be similar.
We relate our proposal to similar estimators by demonstrating that SCM is a regularized inverse propensity score weighting (IPW) estimator, using pre-treatment outcomes as covariates and penalizing the propensity score coefficients via a ridge penalty. This equivalence, which has not previously been noted, relies on a dual representation of the SCM constrained optimization problem. From this perspective, augmented SCM is analogous to augmented IPW in cross-sectional settings (Robins et al., 1994). The IPW perspective also allows us to draw upon the large propensity score literature to better understand SCM and especially to clarify ambiguity about inference and testing.
We make several additional contributions. First, we characterize the finite sample bias for SCM and highlight the role of covariate imbalance. In particular, Abadie et al. (2010) show that SCM is (asymptotically) unbiased under an assumption that SCM weights yield exact balance on the lagged outcomes. This is a strong assumption, however; such weights exist if and only if the treated unit's pre-treatment time series is inside the convex hull of the control units' time series. Similar to other matching and weighting estimators (Robins and Ritov, 1997;Abadie and Imbens, 2011), SCM is therefore subject to the curse of dimensionality -the probability that exact balancing weights exist vanishes as the number of time periods grows (Ferman and Pinto, 2018). Recognizing this, we bound the bias due to covariate imbalance for general weighting estimators under a linear factor model; the results in Abadie et al. (2010) are a special case of this bound. We also propose estimating this bias directly using an outcome model. While we advocate using this to de-bias standard SCM, the estimated bias itself is a useful diagnostic for researchers using SCM without augmentation.
Second, we show that, unlike SCM, ASCM extrapolates outside the convex hull of the control units, possibly leading to negative weights. This ensures much closer balance, reducing bias, but rests more heavily on modeling assumptions, such as linearity. We give theoretical results when the outcome model is ridge regression, and show that covariate imbalance and the corresponding bias will generally be lower for ridge-augmented SCM than for either SCM or ridge regression alone. We also show that the ASCM framework can incorporate flexible outcome models, including panel data methods like matrix completion  and the generalized synthetic control method (Xu, 2017), and off-the-shelf machine learning models like random forests and neural networks.
In addition, ASCM can accommodate auxiliary, time-invariant covariates to further reduce bias.
Despite these advantages, we recommend that, as with any model-based estimator, users devote extra effort to checking model specification, especially in settings where the ASCM and SCM weights yield different estimates.
Third, we draw on the IPW connection to clarify inference and testing for SCM. Abadie et al. (2010Abadie et al. ( , 2015 propose a widely adopted testing procedure for SCM based on a uniform permutation approach. Firpo and Possebom (2017) interpret this test as a Fisher randomization test, though Abadie et al. (2015) interpret it as a placebo test that does not require randomization for validity.
The connection between SCM and IPW suggests that the randomization-based interpretation is a natural one, though other interpretations are possible. From this perspective, however, a uniform permutation test will be invalid -a valid randomization test should weight permutations by the propensity score (Branson and Bind, 2018). Unfortunately, a weighted permutation test is not practical in most SCM settings. We instead propose a model-based inference procedure that is generally conservative, and which performs well in simulations.
Fourth, we contribute to the growing literature on approximate balancing weights in high dimensions (Athey et al., 2018;Tan, 2018;Wang and Zubizarreta, 2018). This literature has focused on settings with sparsity. We extend results to settings with a latent factor model, which motivates using L 2 penalties and ridge regression, rather than L ∞ and Lasso as in existing approaches. Our development will therefore be useful beyond SCM. Finally, we implement the proposed methodology in the augsynth package for R, available at github.com/ebenmichael/augsynth. The paper proceeds as follows. Section 2 introduces notation and the SCM estimator. Section 3 introduces Augmented SCM and characterizes covariate balance for the special case of ridge ASCM.
Section 4 bounds the bias under a linear factor model, the standard setting for SCM, showing that ASCM will generally lead to lower bias than SCM alone. Section 5 demonstrates the equivalence of SCM and IPW. Section 6 discusses testing and inference. Section 7 extends the ASCM framework to incorporate auxiliary covariates. Section 8 reports on numerical illustrations as well as extensive simulation studies. Finally, Section 9 discusses some outstanding issues and possible directions for further research. The appendix includes all of the proofs, as well as additional derivations and technical discussion.

Related work
SCM was introduced by Abadie and Gardeazabal (2003) and Abadie et al. (2010Abadie et al. ( , 2015 and is the subject of an extensive methodological literature. We briefly highlight three relevant strands of research. The first strand assesses the performance of the original SCM estimator under different settings. Kreif et al. (2016), Gobillon and Magnac (2016), Wan et al. (2018), among others, assess the general performance of SCM methods. Botosaru and Ferman (2017) and Kaul et al. (2018) explore the role of auxiliary covariates in SCM and note several pathologies. Ferman and Pinto (2018) consider the behavior of SCM when the weights fail to exactly balance the lagged outcomes, showing that the resulting SCM weights do not converge to oracle weights when the number of time periods grows. Powell (2018) similarly explores variants of SCM without exact balance.
The second strand explores inference and testing for SCM. Abadie et al. (2010Abadie et al. ( , 2015 propose a placebo-based testing approach with a uniform permutation. Firpo and Possebom (2017) interpret this approach as a randomization-based test of the sharp null hypothesis of no impact and derive conditions under which this is valid from a finite sample perspective. Toulis and Shaikh (2018) also take a randomization-based perspective, concluding that the proposed permutation procedure cannot be interpreted as a valid randomization test. Alternatively, Hahn and Shi (2017) assess the same procedure from the perspective of a placebo test, arguing that the approach fails to control Type I error. Ando and Sävje (2013) similarly evaluate this approach under an exchangeability assumption, which they argue is unlikely to be satisfied in practice. Chernozhukov et al. (2017) consider alternative permutations that instead exploit the time series structure of the problem.
Finally, several papers consider a sampling approach to inference Robbins et al., 2017;Imai et al., 2018).
The third strand extends SCM to allow for more robust estimation and for new data structures.
Building on a suggestion in Abadie et al. (2015), several papers have connected SCM to penalized regression Abadie and L'Hour, 2018;Minard and Waddell, 2018). Hazlett and Xu (2018) instead outline a promising approach for improving SCM estimation by first using a kernel approach to transform the raw lagged outcomes. There have also been several proposals to use outcome modeling rather than SCM-style weighting. These include the matrix completion method in , the generalized synthetic control method in Xu (2017), and the combined approaches in Hsiao et al. (2018). Relatedly, Doudchenko and Imbens (2017) relax the SCM restriction that control unit weights be non-negative, arguing that there are many settings in which negative weights would be desirable.
Finally, our work also builds on the recent literature on balancing weights, also known as calibrated propensity scores. This literature modifies the traditional propensity score estimator and instead estimates weights that directly balance covariate means between treated and control units, rather than balancing them indirectly by first estimating the propensity score coefficients.
2 Overview of the Synthetic Control Method

Notation and setup
We consider the canonical SCM panel data setting with i = 1, . . . , N units observed for t = 1, . . . , T time periods. Let W i be an indicator that unit i is treated. We assume that all treated units receive the treatment at a common time T 0 < T ; units with W i = 0 never receive the treatment. There are a total of N 1 = i W i treated units and N 0 = N − N 1 control units, often referred to as donor units in the SCM context. The outcome is Y and is typically continuous. 1 We adopt the potential outcomes framework (Neyman, 1923;Rubin, 1974) and invoke SUTVA, which assumes a well-defined treatment and excludes interference between units (Rubin, 1980). The potential outcomes for unit i in period t under control and treatment are Y it (0) and Y it (1), respectively. 2 Observed outcomes are: (1) To keep notation simple, we assume that there is only one post-treatment observation, T = T 0 + 1, though our results are easily extended to larger T − T 0 . We therefore use Y i to represent the single post-treatment observation for unit i, dropping t from the subscript. We use X it , for t ≤ T 0 , to represent pre-treatment outcomes, which emphasizes that pre-treatment outcomes serve as covariates in SCM. With some abuse of notation, we use X 0· to represent the N 0 -by-T 0 matrix of control unit pre-treatment outcomes and Y 0 for the N 0 -vector of control unit post-treatment outcomes. For most of our discussion, we will restrict attention to the case where only a single unit, i = 1, is treated; that is, N 1 = 1 and N 0 = N − 1. Thus, Y 1 is a scalar, and X 1· is a T 0 row vector of treated unit pre-treatment outcomes. 3 The data structure is then: 1 The source of randomness varies across the SCM literature. As we discuss in Section 4, we focus on the setting where the units are fixed and uncertainty comes from noisy realizations of the latent factor model. This is in contrast to randomization inference in which treatment assignment is the only source of randomness.
2 To simplify the discussion, we assume that potential outcomes under both treatment and control exist for all units in all time periods t > T0. With some additional technical caveats, we could relax this assumption to only require that Yit(0) be well-defined for all units, without also requiring control units to have well-defined potential outcomes under treatment.
3 With multiple treated units treated at the same time, we can overload Y1 and X1· to denote averages across

Synthetic Control Method
The Synthetic Control Method imputes the missing potential outcome for the treated unit, Y 1 (0), as a weighted average of the control outcomes, Y 0 γ. Abadie and Gardeazabal (2003) and Abadie et al. (2010Abadie et al. ( , 2015 propose choosing weights γ as a solution to the constrained optimization problem: where · 2 2 is the squared 2-norm on R T 0 , T 0 t=1 (X 1t − X 0t γ) 2 , and the constraints limit γ to the N 0 simplex, γ ∈ ∆ N 0 .
Equation (3), known as the constrained regression formulation of SCM, follows the recent methodological literature and focuses solely on balancing the lagged outcomes X (see Doudchenko and Imbens, 2017;Ferman and Pinto, 2018;Powell, 2018). By contrast, the original SCM formulation also includes auxiliary covariates Z and a weighted L 2 norm. As Kaul et al. (2018) and others have shown, (3) is identical to the original SCM proposal in a range of practical settings.
We discuss additional complications that arise in the original SCM formulation in Section 7.

SCM generally fails to achieve exact balance
The SCM weights in Equation (3) minimize the imbalance of pre-treatment outcomes between the treated unit and synthetic control. Abadie et al. (2010) show that the resulting estimator is unbiased (asymptotically in T 0 ) under an assumption that exact balance can be achieved, that is, X 1t − X 0tγ scm = 0 for all t. This is a strong assumption, however. Weights that yield exact balance exist if and only if the treated unit is inside the convex hull of the control units.
treated units. The implicit estimand is the (Sample) Average Treatment Effect on the Treated. We return to this point in the discussion in Section 9.
Achieving exact balance is therefore subject to the curse of dimensionality (Robins and Ritov, 1997;Abadie and Imbens, 2011;D'Amour et al., 2017). Informally, the probability that exact balancing weights exist vanishes as the dimension of the covariates, T 0 , grows large. For example, under a population model, Zhao and Percival (2017) show that the probability that the treated unit is in the convex hull of the control units decreases exponentially in the number of covariates. For exact balancing weights to exist (with high probability), the number of control units must therefore be exponentially larger than T 0 , a far cry from the typical SCM setting with N ∼ T 0 . Ferman and Pinto (2018) investigate this question in the context of SCM under a linear factor model, arguing that the SCM weights fail to exactly balance the underlying factors with infinite T 0 . 4 As with other matching or weighting estimators, failing to balance covariates can introduce and ε it is mean-zero noise conditionally independent of treatment assignment. Then the bias for a weighting estimator is m(X 1 ) − W i =0γ i m(X i ). This is zero if m(X) is linear with respect to X and the weights achieve exact balance, but may be non-zero if either condition is not satisfied.
For instance, as we show in Section 4, under a linear factor model, m(X) is approximately linear for large T 0 . At the same time, achieving exact balance is particularly challenging for large T 0 , suggesting that SCM will be biased. In the next section, we propose to estimate this bias directly and then use this estimate to de-bias the original SCM estimate.

Overview
Our main contribution is to propose an Augmented SCM (ASCM) estimator that combines SCM with outcome modeling. Specifically, letm(X i ) be an estimated outcome model under control.
Thenm(X 1 ) − W i =0γ im (X i ) is an estimate of the SCM bias. We propose the following bias- 4 We confirm this intuition with simulation. In one simple simulation, presented in Figure 1, the probability of achieving exact balance drops from 47 percent with T0 = 4 to 1 percent with T0 = 9. In "calibrated" simulations, presented in Section 8, there is not a single Monte Carlo draw in which SCM achieves exact balance.
5 One approach to mitigating bias is to proceed with the analysis only if covariate imbalance is small. Specifically, Abadie et al. (2015) recommend against using SCM when "the pre-treatment fit is poor or the number of pretreatment periods is small." There is little guidance about what constitutes poor fit, however, and common practice is fairly ad hoc. An estimate of the bias based on a prognostic scorem(·) is a natural summary for assessing quality of fit. However, while this pre-screening might be an effective approach for a specific application, it could lead to a file drawer problem. In our simulations in Section 8, we find that a range of methods give excellent performance when we condition on cases with good covariate balance, but that this conditioning substantially limits the utility of the methods. While SCM does poorly when covariate balance is poor, many alternatives (including ASCM) are more robust to imbalance.
reducing estimator for Y 1 (0): whereγ are SCM weights. This specializes to standard SCM when we setm(·) to be a constant.
Equations (4) and (5), while equivalent, highlight two distinct motivations for ASCM. Equation (4) directly corrects the SCM estimate, This is analogous to bias correction for inexact matching (Abadie and Imbens, 2011). If the estimated bias is small, then the SCM and ASCM estimates will be similar. Equation (5) is analogous to Augmented IPW (Robins et al., 1994), which begins with the outcome model but uses SCM to re-weight the residuals. This is comparable in form to the generalized regression estimator in survey sampling (Cassel et al., 1976;Breidt and Opsomer, 2017), which has been adapted to the causal inference setting by, among others, Athey et al. (2018) and Tan (2018). We develop the IPW analogy further in Section 5.  Brodersen et al., 2015). Alternatively, we can use generic supervised learning methods, such as random forests and neural networks. The ASCM framework also nests recent proposals from Doudchenko and Imbens (2017) and Ferman and Pinto (2018) for "de-meaned SCM" or SCM with an intercept shift, which correspond to a simple unit fixed effects outcome model, m(X i ) = α i . Ferman and Pinto (2018) show that this estimator dominates both standard difference-in-differences and SCM, asymptotically in T 0 . We explore these options via simulation in Section 8. Finally, in Section 7, we generalize ASCM to include additional covariates beyond lagged outcomes.

Ridge-augmented SCM
We now explore the special case of ASCM where the outcome model m(X) is fit with (linear) ridge regression. Using results from survey sampling, we first show that we can write this estimator as a single weighting estimator (possibly with negative weights) that adjusts the SCM weights to allow for better balance. Using this equivalence, we then show that ridge-augmented SCM generally has better covariate balance than either SCM or ridge regression alone, though it may have higher variance. While we restrict our theoretical results to ridge-augmented SCM, we anticipate that these can be extended to more general, non-linear outcome models via the model calibration weights framework of Wu and Sitter (2001).
Letm(X i ) = X i·η , whereη are the coefficients of a ridge regression of control post-treatment outcomes Y 0 on centered pre-treatment outcomes X 0· with penalty hyper-parameter λ. Ridge regression, like OLS, has a closed-form expression as a weighting estimator (see Appendix A.1).
Thus, the augmented SCM estimator merely combines two different weighting estimators, SCM and ridge regression.
Lemma 1. The ridge-augmented SCM estimator specialization of (5) is: Lemma 1 shows that ASCM weights -which, unlike SCM weights, can be negative -adjust the raw SCM weights to achieve better covariate balance. When SCM weights exactly balance the lagged outcomes, ridge ASCM and SCM weights are equivalent, and when SCM yields good balance or the tuning parameter λ is large, the estimated bias is small and the two weights are close to each other. Conversely, when SCM has poor balance and λ is small, the adjustment will be large and the weights will be far apart. 6 As a result, ridge-augmented SCM weights will generally achieve better pre-treatment fit than weights from SCM alone, although at the cost of higher variance.
Lemma 2. Let σ 2 min be the minimum eigenvalue of the sample covariance matrix 1 N 0 X 0· X 0· . Then with κ = λ N 0 σ 2 min + 1 > 1, the ridge-augmented SCM weightsγ aug satisfy: and Since κ−1 κ < 1, ridge-augmented SCM will have strictly better covariate balance than SCM alone, except in the special case of exact balance. At the same time, ridge-augmented SCM weights will generally have larger variance than pure SCM weights, with larger discrepancies for worse SCM imbalance. Intuitively, this larger variance arises because, unlike SCM weights, ASCM weights can be negative, which can increase the spread of the weights.
Allowing for negative weights is an important departure from standard SCM: Abadie et al. (2010Abadie et al. ( , 2015 argue that negative weights are undesirable because they are difficult to interpret and 6 In practice we choose λ using K-fold cross-validation for the outcome ridge regression. allow for extrapolation. 7 The SCM non-negativity constraint implies choosing a synthetic control within the convex hull created by the control units, even when the treated unit lies outside this hull, and the distance from the treated unit to the convex hull creates the potential for bias. Thus, even with moderate dimensional covariates, an estimator that constrains weights to be non-negative will be biased in practice. ASCM, by contrast, uses negative weights to extrapolate outside of the convex hull, ensuring much closer balance but resting more heavily on the assumption that the expected value of Y 1 (0) is (approximately) linear in the control outcomes. We confirm this intuition with simulations in Section 8. In cases where covariate balance is excellent, there is little penalty to restricting weights to be non-negative; otherwise, this constraint can lead to severe bias, at least relative to ASCM. Doudchenko and Imbens (2017) make a similar point with an analogy to bias correction for matching, arguing that negative weights play an important role in reducing bias when exact matches are infeasible.
Finally, in the appendix we compare ridge-augmented SCM to ridge regression alone, again relying on a representation of ridge regression as a weighting estimator. As we show, the ridge regression weights correspond to a special case of the elastic-net synthetic controls estimator proposed by Doudchenko and Imbens (2017), with the elastic-net parameter set to zero, and to a penalized version of the Oaxaca-Blinder weights considered by Kline (2011). Ridge regression achieves worse balance than ridge ASCM, but yields lower sampling variance. In the special case with no regularization, ridge regression weights reduce to the standard regression weights discussed in Abadie et al. (2015), and both standard regression and ridge ASCM will yield perfect balance.

Bias under a linear factor model
We now characterize the bias under the linear factor model considered in Abadie et al. (2010), with treatment assignment that is ignorable given unobserved factor loadings. We begin with bias for a general weighting estimator and then turn to SCM, showing that the results in Abadie et al. (2010) are a special case of our bound. Finally, we show that ridge ASCM will generally have a tighter bias bound than SCM alone. We confirm these results with simulations in Section 8.

Bias for a general weighting estimator
Following the setup in Abadie et al. (2010), we assume that there are J latent time-varying factors µ t = {µ jt }, j = 1, . . . , J, with max jt |µ jt | ≤ M , where J will typically be small relative to N . Each unit has a vector of factor loadings φ i ∈ R J . Control potential outcomes are weighted averages of these factors plus additive noise ε it : where the only random quantities are the noise terms ε it . Slightly abusing notation, we collect the pre-intervention factors into a matrix µ ∈ R T 0 ×J , where the t th row of µ contains the factor values at time t, µ t .
Following Bai (2009) and Xu (2017), we assume that the factors are orthogonal and normalized, i.e. that 1 T 0 µ µ = I J , and that treatment assignment W i is ignorable given a unit's factor loadings, where the expectation is taken with respect to ε T = (ε 1T , . . . , ε N T ). We further assume that the error terms ε it are independent (across units and over time) sub-Gaussian random variables with scale parameter σ.
Under the linear factor model and ignorability given φ i , an estimator that balances φ i will yield an unbiased estimate of Y 1 (0), and if exact balance is not achieved the bias will be proportional to the level of imbalance in φ i . However, ignorability given the latent factors φ i does not generally imply ignorability given the observable X i , and ensuring balance in X i will not necessarily ensure balance in φ i . We show in the appendix that the bias of any weighting estimator with weights γ can be expressed as The first term is the imbalance of observed lagged outcomes and the second term is an approximation error arising from the latent factor structure. Using our assumption that the noise is sub-Gaussian, we can bound this bias using the triangle inequality.
This result holds for any weights that sum to one, and does not require non-negativity. The approximation error is generally non-zero since the outcome is linear in φ rather than linear in X; however, with large T 0 the approximation error will be small. Interestingly, this term depends on the L 2 norm of the weights, γ 2 , which does not generally enter into bias bounds in simpler settings. Similar results exist for bias in other panel data settings. For instance, the approximation error in Equation (12) is analogous to so-called " Nickell (1981) bias" that arises in short panels.

Bias for SCM
To apply Theorem 1 to SCM, we consider two cases: the special case with exact balance and the more general case with approximate balance. In the exact balance case, the first term of (12) is zero and the second term goes to zero, with high probability, as T 0 → ∞. This is the basis of the Abadie et al. (2010) claim that SCM is asymptotically (nearly) unbiased in a factor model with exact balance. Intuitively, the lagged outcome for unit i at time t, X it , is a noisy proxy for the index φ i µ t . Thus, as we observe more X it -and can exactly balance each one -we are better able to match on this index and, as a result, on the underlying factor loadings.
If we do not assume exact balance, the bias bound contains two competing terms: the approximation error, which is decreasing in T 0 , and the imbalance in X, which is non-decreasing in T 0 . Figure 1 illustrates this tension via simulation. 8 As T 0 increases, the average covariate imbalance (in terms of RMSE in X) also increases, eventually leveling off. The bias decreases initially as the approximation error falls, but levels off to a positive value as imbalance comes to dominate.
While Theorem 1 only provides an upper bound on the bias, rather than an expression for the bias itself, we nonetheless argue that SCM will still be biased with large T 0 . Ferman and Pinto (2018) show that -even if weights exist that exactly balance the latent factor loadings -as T 0 → ∞ the SCM weights will converge to a solution that does not exactly balance the latent factors, leading to bias. Theorem 1 complements their asymptotic analysis with a finite sample bound that holds for all weighting estimators, and explicitly includes the level of imbalance in the lagged outcomes. In Appendix A.3 we discuss further connections to their results and the duality between SCM and IPW presented in the following section. Our simulation evidence, both in Figure   1 and in Section 8, is consistent with conclusion that SCM is biased in a range of scenarios.

Bias for Augmented SCM
Lemma 2 shows that ridge ASCM will generally have better covariate balance than SCM. We now combine this with Theorem 1 to show that ridge ASCM will also have a tighter bias bound than SCM alone. While our theoretical discussion is again limited to bounds on the bias, simulations in Section 8 confirm that the realized bias is consistently smaller for ridge ASCM than SCM alone, at least in the settings we consider.
As in the general case in Theorem 1, the level of imbalance and the complexity of the weights play possibly competing roles. Specifically, the regularization parameter λ (or the transformation κ) controls how much to prioritize balance. In the extreme case with λ = ∞, ridge ASCM is equivalent to SCM. As we reduce λ, the imbalance in X decreases but γ aug 2 increases, which increases the approximation error. Note that if X 0· is full rank (requiring N > T 0 ), ridge ASCM weights exists with λ = 0 and ridge ASCM exactly balances the lagged outcomes, even where non-augmented SCM does not.
Finally, while the constant terms in Theorem 1 and Corollary 1 are relatively loose, it is instructive to consider the bias bounds for SCM and ridge ASCM when the number of control units is large relative to the number of latent factors. In this case, the approximation error is small and it is bias-optimal to minimize imbalance in the lagged outcomes, setting λ to be small. Thus, the bias bound for ridge ASCM will be lower than the corresponding bound for SCM. As we show through simulation in Section 8, this intuition holds even when the number of control units is only slightly larger than the number of latent factors, and even in this case the reduction in bias outweighs any increase in variance.

Connection to other balancing estimators
We now connect SCM to other balancing estimators, which builds intuition, helps to motivate the Augmented SCM estimator proposed in Section 3, and gives additional clarity for inference and testing. First, we notice that a form of SCM, which we call penalized SCM, is a special case of an approximate covariate balancing weights estimator. Second, we extend existing results on the duality between balancing weights and inverse propensity score weights to show that SCM is indeed a form of inverse propensity score weighting.

Penalized SCM
There may be no unique solution to the original SCM problem in Equation (3), if multiple sets of weights achieve exact balance on the lagged outcomes. Following Abadie et al. (2015) and Doudchenko and Imbens (2017), we modify the original SCM procedure to penalize the dispersion of the weights with a strongly convex dispersion function, f (γ i ). For a sufficiently small penalty, penalized SCM will be nearly identical to standard SCM in cases where the latter has a unique solution. However, the penalized SCM problem is guaranteed to have a unique solution for any positive penalization, which is analytically convenient.
To fix ideas, we consider the entropy penalty, f (γ i ) = γ i log γ i , used in Robbins et al. (2017). 9 9 Many other dispersion functions are possible, including an elastic net penalty , a measure of pairwise distance (Abadie and L'Hour, 2018), and a measure of outcome variance (Minard and Waddell, 2018); we discuss alternative dispersion penalties in Section 5.3. An additional motivation for penalized SCM is that the standard SCM weights can often be unstable, with very few (often just three or four) donor units receiving positive weights. While this may minimize bias, it has high variance; in practice, researchers might want to accept some bias for lower variance.
Specifically, the entropy penalized SCM weights solve: The hyperparameter ζ > 0 sets the relative priority of minimizing the entropy term, which penalizes very large and very small weights, versus covariate imbalance. A larger ζ means greater covariate imbalance, and thus more bias, but lower variance due to the more dispersed weight vector. As with other regularized estimators, if a unique solution exists to the un-regularized (3), we can find an arbitrarily close solution to (14) by setting ζ sufficiently small.
With this setup, we can immediately see that entropy balancing (Hainmueller, 2011) is a special case of Equation (14) in which the weights yield exact balance. Thus, entropy penalized SCM can also be motivated as entropy balancing with approximate-rather than exact-balance. More broadly, SCM is one example of a broad class of covariate balancing weights that have recently become more prominent. In addition to entropy balancing, examples include stable balancing weights (Zubizarreta, 2015), approximate residual balancing (Athey et al., 2018), and several others (Tan, 2017;Wang and Zubizarreta, 2018;Zhao, 2018

SCM is regularized IPW
Once we recognize that SCM yields covariate balancing weights, we can leverage recent results connecting balancing and propensity score weights to show that SCM is a form of inverse propensity score weighting. First, we show the duality between entropy-penalized SCM with exact balance and unregularized IPW. This result follows Zhao and Percival (2017), though our argument is more direct. Second, we relax the restriction that the weights yield exact balance. In this case, SCM is equivalent to regularized IPW. We extend results from Wang and Zubizarreta (2018) to the case of an L 2 norm and show that entropy-penalized SCM maps to a ridge penalty on the propensity score coefficients. In the appendix we derive a general duality between balancing weights and propensity score estimation that encompasses many estimators in the literature, including extensions to other link functions.

Penalized SCM with exact balance is unregularized IPW
We consider the penalized SCM estimator in Equation (14). Enforcing exact balance, this estimator can be rewritten as: min Note that the entropy dispersion penalty includes the SCM non-negativity constraint. In the appendix, we show that this problem has an equivalent Lagrangian dual: where β and α are the Lagrange multipliers on the two constraints in (15). The solution to (16), {α,β}, corresponds to weights on the control units: With this link between the primal problem (15) and the Lagrangian dual (16), we can see that the exact balance constraints in the primal correspond to the first order conditions of the dual: Thus, the donor weightsγ have the form of IPW weights with a logistic link function, where the propensity score is π(x i ) = logit −1 (α+β x i ) and the odds of treatment are π(x i ) 1−π(x i ) = exp(α+β x i ) = γ i . Importantly, the parameters in Equation (17) are fit via calibrated estimation rather than the two-step, maximum likelihood-based approach that is standard for IPW but is impractical in typical SCM settings. 10 Specifically, the Lagrangian dual (17) fits the propensity score coefficients so that the implied odds of treatment, γ i = exp(α + β X i· ), lie on the simplex and the weighted mean of the control units' covariates is exactly equal to the value of the treated unit, W i =0 X i γ i = X 1 .
10 In traditional IPW, the propensity score model is first estimated via maximum likelihood, then the estimated propensity scores e(Xi) are used to form IPW weights. McCullagh and Nelder (1989) and King and Zeng (2001) show that the MLE for logistic regression can be badly biased when the response is a rare event. Since 1 − e(Xi) is in the denominator, the re-weighting step can amplify this bias. Calibrated propensity score estimation also yields consistent estimates of the true propensity score under appropriate conditions; see, for example, Zhao and Percival (2017) and Tan (2017). (2017) and Wang and Zubizarreta (2018) for additional discussion.

Penalized SCM with approximate balance is regularized IPW
We now relax the unrealistic constraint that SCM yield exact balance. In the dual perspective of balancing weights as IPW, allowing for approximate balance is equivalent to regularizing the propensity score model. Specifically, analogous to the argument under exact balance, the Lagrangian dual to the entropy-penalized SCM problem (14) includes a propensity score model that now includes a ridge penalty on the propensity score coefficients: In this form it is clear that ζ controls the level of regularization of the propensity score parameters, which maps back to the weights γ i . When ζ is large, the parameter estimatesβ will be near zero, implying that the weights will be near uniform. Conversely, when ζ is small,β may be large in magnitude, allowing for extreme weights that prioritize lower bias at the price of higher variance. In practice, SCM implicitly chooses ζ to be as small as possible, so weights are extreme; it is common for only three or four units to receive positive weights. See Doudchenko and Imbens (2017) for a discussion of choosing ζ by cross validation.

General primal-dual connection
Finally, we briefly characterize the entropy-penalized SCM problem as a special case of a broader class of problems. Specifically, we can re-write the penalized SCM problem in Equation (14) in a more general form (see Ben-Michael et al., 2018, for additional discussion): This general form has two key components: f is a strongly convex measure of dispersion (e.g. entropy), which guarantees uniqueness, and h ζ is a measure of distance (e.g. the L 2 norm). This formulation covers several estimators, including penalized SCM, entropy balancing (Hainmueller, 2011), Oaxaca-Blinder weights (Kline, 2011), and minimal approximately balancing weights (Wang and Zubizarreta, 2018). In the appendix we extend the above arguments to derive the Lagrangian dual of this general balancing weights problem (19): where a convex, differentiable function g has convex conjugate g * (y) ≡ sup x∈dom(g) {y x − g(x)}.
The two components of the primal problem (19) control the propensity score model and how it is regularized. The dispersion measure f (·) determines the link function of the propensity score model, where the odds of treatment are π(x) 1−π(x) = f * (α + β x). Note that un-penalized SCM, which can yield multiple solutions, does not have a well-defined link function. The balance criterion h ζ (·) determines the type of regularization through its conjugate h * ζ (·). This formulation recovers the duality between entropy balancing and a logistic link (Zhao and Percival, 2017), Oaxaca-Blinder weights and a log-logistic link (Kline, 2011), and L ∞ balance and L 1 regularization (Wang and Zubizarreta, 2018). This more general formulation also suggests natural extensions of both SCM and ASCM beyond the L 2 setting to other forms, especially L 1 regularization.
6 Testing and inference 6.1 Permutation-based testing Abadie et al. (2010Abadie et al. ( , 2015 propose a widely adopted testing procedure for SCM based on a uniform permutation test. Interpretation of this test, however, is unsettled. Abadie et al. (2015) justify it as a placebo test that does not require randomization for validity. 11 Firpo and Possebom (2017) instead justify it as a Fisher randomization test with uniform treatment assignment and then assess sensitivity to this assumption. 12 The IPW equivalence suggests that the randomization-based perspective is a natural interpretation. A valid randomization-based test for an IPW estimator, however, requires weighting the permutation distribution by the propensity score. In the appendix, we describe valid testing with known, varying propensity scores (see Branson and Bind, 2018), and show that the uniform permutation test differs from this ideal case. Thus, we argue that uniform permutation testing will generally be invalid from the randomization-based perspective, though other perspectives are possible.
A weighted permutation test using the estimated propensity score is promising in principle. Toulis and Shaikh (2018) give conditions under which the corresponding p-values will be valid asymptotically. In practice, however, there is little reason to expect those conditions to hold in 11 Hahn and Shi (2017) argue that the placebo test interpretation only holds under strong assumptions on both the model and outcome distribution. Ando and Sävje (2013) show that a similar motivation, which requires exchangeability of the test statistic, is unlikely to hold since units on the border of the convex hull of the factor loadings are systematically different from the treated unit, which is typically assumed to be inside the convex hull.
12 Chernozhukov et al. (2017) propose an alternative permutation approach that permutes the time periods rather than the treatment assignment. The key assumption for the validity of this approach is unbiasedness of the estimator, which, as we argue, is likely violated in practice for SCM. Simulation results in the appendix confirm that this approach is invalid in the settings we consider. typical, finite sample SCM settings, with a heavily regularized propensity score based on a single, treated unit. Simulation evidence presented in the appendix confirms this pessimistic outlook.
Moreover, even if this approach were valid, it would be impractical in most SCM settings: since SCM weights are generally sparse, few permutations will have non-zero weight, and the p-value distribution under both the null and alternative will be pathological. For instance, it is quite common for SCM weights to be positive for only three or four donor units. If four units each get equal weight, the lowest possible p-value is p scm = 0.25.

Model-based inference
As an alternative to permutation testing, we propose a model-based approach to inference based on the placebo distribution and show that the resulting inference is conservative under some assumptions on the error distribution. Specifically, we consider a generic outcome model with independent, sub-Gaussian noise ε it , Y it (0) = m it + ε it . To simplify exposition and notation, we initially assume that the noise terms ε it are homoskedastic at each time t, with common variance ν t = Var( it ).
Under this model, the variance of the SCM or ASCM treatment effect estimate is: where ν T is the residual variance at post-treatment time T and where the variance conditions on the observed lagged outcomes and treatment assignment. With uniform weights, Equation (21) reduces to the usual variance of the difference in means under homoscedasticity.
We propose to estimate the noise variance ν T via the average squared placebo gap: where Y i is the leave-one-out SCM estimate of Y i , j =i γ (i) j Y obs j . Importantly,ν T is a conservative estimator for ν T , in the sense of having a positive bias. To see this, note that the variance of placebo gaps is strictly larger than the variance of the noise: We then estimate the sampling variance ofτ as Proposition 1 shows that this is a conservative estimate for Var(τ ).

Proposition 1. Let Bias
Under the linear factor model (10), Var(τ ) is conservative: Proposition 1 shows that the upward bias of Var(τ ) depends on the bias of the placebo estimates and the L 2 -norm of the placebo weights. 13 Building on results from Section 4, these quantities will be small when the control units are similar to each other and tightly packed together, which will limit both the bias and the spread of the weights. Conversely, the upward bias of Var(τ ) will be large if there are extreme outliers in the control group.
Many modifications to the simple variance estimate in (23) are possible. For example, we can divide the i th squared placebo gap by its contribution to the bias, 1 + γ (i) 2 2 . We can similarly extend to the fully heteroskedastic case by weighting the mean squared placebo gap. In addition, we can extend this variance estimate to the setting with multiple post-treatment time periods by separately estimatingν t for each post-treatment time period and plugging in to (23).
While we advocate the model-based inference approach, there are many possible alternatives.
First, the bootstrap is an attractive approach for inference for IPW estimators in cross-sectional settings (Funk et al., 2011). The SCM setting, however, falls between traditional IPW, where standard case-resampling bootstrap works well, and matching-style estimators, where it can fail (Abadie and Imbens, 2008). The simulations in Section 8 include a simple non-parametric bootstrap and show that it has poor coverage. Second, Robbins et al. (2017) propose estimating standard errors following a survey sampling approach. This method, however, is restricted to estimators with non-negative weights. Finally, we could pursue a design-based approach to inference, which follows naturally from both the IPW perspective and the randomization-based testing approach for SCM (e.g., Doudchenko and Imbens, 2017). Our initial simulations, however, found that both the design-based and survey sampling approaches performed poorly in settings with a single treated unit. We leave a thorough investigation to future work. 7 Incorporating auxiliary covariates 7.1 Extending Augmented SCM So far, we have focused on a simplified version of SCM that uses only pre-treatment outcomes as covariates. The original SCM formulation, however, also includes auxiliary covariates Z i ∈ R d , which are typically time invariant and which can also include functions of the lagged outcomes, such as the pre-treatment average. We assume these covariates are centered by the control unit averages, and for now assume that the matrix of control unit covariates, Z 0 , is full rank. Abadie

et al. (2010) propose a variant of Equation (3) that minimizes a weighted imbalance measure in Z,
with weights chosen to minimize imbalance of the lagged outcomes X. Using our IPW perspective, we show in Appendix A.4.1 that this approach fits a (calibrated) propensity score model using Z as covariates and a weighted ridge penalty, with weights chosen to minimize the imbalance in X. 14 Our results suggest three transparent alternatives for incorporating auxiliary covariates. First, we can expand ASCM to include Z alongside X, both in the SCM balance criterion and in the outcome model, m(X i , Z i ). This is natural from the (augmented) IPW perspective; including Z in the balance criterion is equivalent to including it in the propensity score equation. See Appendix A.4.2 for additional details, including the possibility of giving different weights to X and Z in the SCM balance criterion and the outcome model regularization.
Second, we can partition the variables in the ASCM framework, using SCM to balance the lagged outcomes and the outcome model to adjust for possible bias due to imbalance in the auxiliary covariates. Specifically, this approach uses our ASCM proposal, Equation (5), but expresses the outcome model solely in terms of Z. Because we assume that Z is full rank, we can fit this model To understand this approach, consider linear projections of X and Y onto Z, using coefficients estimated from control observations. We can write the projections asX = HX 0· andŶ = HY 0 , where H = Z(Z 0 Z 0 ) −1 Z 0 , and write the residuals from these projections asX ≡ X −X anď Y ≡ Y −Ŷ , respectively. Lemma 3 indicates that ASCM with OLS on Z as the outcome model is a weighting estimator that exactly balances the covariates Z and thus the projectionsX. The remaining imbalance comes solely from the residual componentX.
Lemma 3. For any weight vectorγ that sums to one, the ASCM estimator from Equation (5) that usesm(Z i ) = Z iη ols as the outcome model is a weighting estimator, and the imbalance in the lagged outcomes and the auxiliary covariates are This has several implications. First, the approach outlined above, usingm(Z i ) = Z iη ols as the outcome model in ASCM, is equivalent to first residualizing Y against Z, then estimating the treatment effect asY 1 −Y 0γ scm , whereγ scm are the SCM weights that minimize imbalance in X.
Second, this suggests an improvement on this approach that replacesγ scm with SCM-or ASCMstyle weights that minimize imbalance inX instead of X. This can be implemented in the form of a partitioned regression: first residualize X and Y against Z, then apply SCM or ASCM to the residualsX andY . By Lemma 3, this perfectly balances Z. Furthermore, by achieving better balance on the residualsX, this approach achieves better balance on X than is obtained by separately balancing the raw lagged outcomes and fitting OLS on the auxiliary covariates. 15 In addition, because we can represent this approach as a weighting estimator, we can compute standard errors with the procedure in Section 6.2.
Finally, the partitioned regression procedure as outlined requires Z 0 to be full rank. If it is not (e.g., if d > N 0 ), we can still follow the partitioned regression procedure, though the residuals will only be approximately orthogonal to the auxiliary covariates, and Equation (27) will not hold. In practice we suggest following the dimension reduction approaches in the following section, and do this in the numerical illustration in Section 8. Alternatively, we can directly model the prognostic score (Hansen, 2008). For example, we could use the ridge regression model in Section 3.2 to estimateŶ i (0) for each unit, then fit SCM using 15 In addition, under the linear factor model with covariates considered by Abadie et al. (2010) and Xu (2017), the bias bound in Theorem 1 will still hold becauseγ cov perfectly balances the covariates. Note that while this partitioned ASCM approach always exactly balances the auxiliary covariates, whether it achieves better balance on the original X than SCM alone, without auxiliary covariates, is data dependent. We thus propose this approach as a sensible default, and as with SCM and ASCM encourage careful balance checking. 16 Doudchenko and Imbens (2017) and Ferman and Pinto (2018) propose applying SCM to de-meaned data Xit −Xi. This can be seen as a special case of our partitioned regression proposal. this as the only covariate. Or, analogous to the V matrix in Abadie et al. (2010), we could weight the covariates by their relative predictive power for Y i (0). 17 Hazlett and Xu (2018) also suggest a promising kernel-based approach in the context of SCM, first proposing a kernel representation of the lagged outcomes and then balancing a low-rank approximation to the kernel.

Dimension reduction
We evaluate specific implementations of dimension reduction in our simulations in Section 8.
Simulation evidence indicates that SCM using a well-chosen dimension reduction can improve over SCM using the full vector of pre-treatment outcomes, but that ASCM using the full covariate vector and a flexible outcome model generally improves on either.

Simulations and numerical illustrations
We now turn to a numerical illustration and simulations. We focus on two prominent SCM examples: the impact of Proposition 99 on cigarette consumption in California (Abadie et al., 2010) and the impact of terrorism on the Basque economy (Abadie and Gardeazabal, 2003). We describe the Prop. 99 setting below and describe the Basque example in the appendix. We generate simulation studies designed to mimic each application. Overall, we find that SCM is badly biased across these simulations and that there can be substantial gains from ASCM and related approaches. We also use these simulations to show that standard uniform permutation tests are generally invalid and to evaluate coverage of the variance estimator in Section 6.2.

Illustration: California Prop. 99
In their seminal paper, Abadie et al. (2010) apply SCM to the impact of California Proposition 99 on per-capita cigarette sales. Enacted in 1989, Prop. 99 increased cigarette taxes in addition to launching several anti-tobacco projects in the state. The authors find that, for the 1989-2000 period, Prop. 99 reduced annual cigarette sales by roughly 25 percent. A small cottage industry has emerged around re-analyzing this example (e.g., Doudchenko and Imbens, 2017;Hazlett and Xu, 2018;Hsiao et al., 2018). Figures 2 and 3 show the results of our re-analysis of the California data. Figure 2 shows gap plots of the ATT estimates, plus or minus two standard errors calculated using our model-based proposal in Section 6.2. We consider three estimators: (1) SCM alone, 18 (2) ridge-augmented SCM, and (3) the partitioned strategy described in Section 7, using an OLS regression on four auxiliary covariates and the pre-treatment average as the outcome model and balancing the residuals with 17 This is the L 2 analog of the common "covariate screening" approach for high-dimensional covariates. For example, Ning et al. (2017) use a Lasso regression of Y on X as a pre-processing step to select important covariates and then balance that selected subset. See also Belloni et al. (2014).
18 Note that the SCM estimator used here balances all the lagged outcomes. The original Abadie et al. (2010) study used four auxiliary covariates (cigarette prices, per capita income, the share of the population aged 15-24, and per capita beer consumption) and three lagged outcomes (1975, 1980, and 1988) in Z (see Section 7). Perhaps surprisingly, we obtain nearly identical estimates using all the lagged outcomes and no auxiliary covariates. SCM. Figure 3a shows the balance across auxiliary covariates and, in the last row, across pretreatment average outcomes. Recall that by Lemma 3, the third estimator achieves exact balance on the auxiliary covariates. Figure 3b shows the donor unit weights for SCM and ridge ASCM. The covariate-adjusted ASCM has similar-sized standard errors to SCM but an estimated effect of around 13 packs per capita, roughly half the size of the SCM estimate but close to the estimated 12 packs per capita in the original Prop. 99 study (Glantz and Balbach, 2000).
We can use the tools we have developed to understand these differences. The balance in the lagged outcomes and auxiliary covariates, while good, is not exact for SCM, which suggests that the estimates are still likely biased. The augmented estimators substantially improve balance: the baseline ridge augmented estimator achieves nearly perfect balance of the lagged outcomes, but leaves some imbalance of other covariates, and the covariate-adjusted ASCM estimator achieves perfect balance on the auxiliary covariates and roughly half the imbalance on pre-treatment outcomes as SCM. This suggests that bias will be smaller for the augmented estimators, especially the covariate-adjusted estimator, than for SCM alone.
Finally, as anticipated, the weights for ridge ASCM are more variable than the sparse, nonnegative standard SCM weights-the L 2 norm of the ridge ASCM weights is roughly 12 percent larger than that of the SCM weights-and many of the weights are negative. Thus, ridge ASCM is extrapolating outside of the convex hull in order to achieve nearly perfect balance on the pre- treatment outcomes. As a consequence, the estimate relies more heavily on global linearity than SCM alone. In addition, these more variable weights increase sampling variability. Finally, we note that the standard error estimates assume homoskedasticity across units, but not across time periods, and so the standard error widen and tighten at different time points.

Calibrated simulation studies
We now turn to simulation studies in which the true data generating process is known. To make our simulation studies as realistic as possible, we conduct "calibrated" simulation studies (Kern et al., 2016), based on estimates of linear factor models fit to the California (N = 38, T 0 = 18) and Basque (N = 17, T 0 = 20) examples. We use the Generalized Synthetic Control Method (Xu, 2017) to estimate factor models with four latent factors for each application. We then simulate outcomes using the distribution of estimated parameters. We model selection into treatment as a function of the latent factors. Additional details are in Appendix A.6. We also present results from two additional DGPs, each calibrated to estimates from the Basque data: a unit and time fixed effects model, and an auto-regressive model with 4 lags.

Simulation results: estimation
SCM and ridge ASCM First, we explore the role of augmentation using simple outcome models.
For each DGP, we consider five estimators: (1) SCM alone, (2) ridge regression alone, (3) ridge ASCM, (4) fixed effects alone, and (5) SCM augmented with fixed effects. Figure 4 shows the main results. Each panel displays the average absolute bias for all five estimators across all simulations.
In addition, all panels show the absolute bias conditional on the simulation yielding good balance, and (5) CausalImpact (Brodersen et al., 2015). We compare ASCM to the pure outcome models as well as pure SCM. We find wide variation across models, with a random forest model and a two layer feed-forward neural network performing relatively poorly and Lasso and CausalImpact (Brodersen et al., 2015) doing better. In all cases, augmenting SCM with the outcome model reduces the bias in SCM and generally improves on the pure outcome model as well. In particular, even augmenting SCM with a flexible but poorly performing machine learning method results in bias reduction relative to SCM alone.
Dimension reduction Appendix Figure C.7 shows the performance of combining dimension reduction with standard SCM. In general, fitting SCM using either (1) unit long-run averages or (2) the prognostic score fit via ridge regression has much lower bias than fitting SCM on all lagged outcomes, though much more bias than the outcome modeling and ASCM estimates shown in

Simulation results: testing and inference
Testing. Using the Basque and California calibrated simulations, Figure 5 shows the empirical distribution for the p-values from uniform permutation tests, p unif as defined in Appendix A.5, with a placebo gap test statistic, Each plot shows three simulations with varying levels of selection: zero, weak, and strong selection. The pattern is striking. While the uniform permutation p-values are valid under zero selection -when the propensity scores are constant across units -they are clearly invalid when this does not hold. This pattern holds across a large number of simulation settings. 19 19 An interesting feature of our simulations is that the empirical Type I error rate differed substantially by test statistic. We obtained better performance with the RMSPE test statistic proposed by Abadie et al. (2010). We leave a thorough investigation to future work. Inference. Finally, we evaluate the model-based inference method described in Section 6.2 using the Prop. 99 simulation. Appendix Table 1 shows coverage results for the leave-one-out estimate of the standard error as well as for the bootstrap. As anticipated, we find that our proposed variance estimator is conservative, at least in settings with small bias, and that over-coverage is more dramatic when the placebo weights have worse balance. By contrast, the bootstrap badly under-covers across simulation settings.

Discussion
SCM remains a widely popular approach for estimating causal effects in policy settings. Despite this popularity, however, SCM remains poorly understood. It also has a major limitation: in typical applications, SCM fails to exactly balance pre-treatment outcomes, creating bias. We address this by proposing the Augmented SCM estimator, which combines SCM and outcome modeling. By extrapolating outside the convex hull of the control units, ASCM achieves better balance and reduces bias, albeit at the price of additional model dependence. ASCM naturally accommodates additional covariates when available to further reduce bias.
We also offer a new perspective on SCM -that SCM is a form of inverse propensity score weighting -which clarifies some open issues. Many of the ideas we present build off suggestions in the original SCM papers; our new perspective on SCM will hopefully help make these ideas more accessible. We also use this perspective to argue that, at least from a randomization-based perspective, the commonly used uniform permutation test is invalid. We instead propose a modelbased inference approach for both SCM and ASCM that works well in simulations.
While Augmented SCM is promising, the SCM problem is fundamentally challenging and there is no silver bullet. As with any analysis, researchers should assess sensitivity to modeling choices and should be restrained in interpreting results. In particular, ASCM is vulnerable to departures from the outcome model, so sensitivity to this should be carefully explored.
There are several directions for future work. Perhaps most important is to develop comprehensive diagnostics and other model checks for Augmented SCM and, related, to find coherent and valid design-based methods for testing and inference in typical SCM settings. This is difficult since any valid approach must successfully handle both small sample sizes and very noisy propensity score estimates. While our model-based procedure is straightforward, valid design-based inference would be particularly attractive.
A second area for future inquiry is the application of the ASCM framework to settings with multiple treated units. When all treated units are treated at the same time, the IPW perspective suggests that averaging the treated units is a natural strategy. By contrast, recent proposals to fit SCM separately for each treated unit (e.g., Dube and Zipperer, 2015) correspond to fitting a separate propensity score for each treated unit, which can be unnatural. The situation is more complicated with staggered adoption, when units take up the treatment at different times (e.g., Toulis and Shaikh, 2018;Imai et al., 2018). This is an important direction for future work.
A third potential extension is to more complex data structures, such as applications with multiple outcomes series for the same units (e.g., measures of both earnings and total employment in minimum wage studies) or hierarchical data structures with outcome information at both the individual and aggregate level (e.g., students within schools).
Finally, many of the ideas we develop for the panel data setting can be extended to more traditional cross-sectional observational studies. As we note in Section 5.3, in some settings, minimizing an L 2 imbalance will be more attractive than minimizing the standard L ∞ imbalance (Athey et al., 2018;Tan, 2017;Ning et al., 2017).
In the end, SCM remains a powerful tool for researchers studying interventions at the aggregate level, and we believe that Augmented SCM helps to mitigate some of SCM's shortcomings. This remains an inherently challenging problem -in many cases, we are trying to impute the missing potential outcome for a single unit -and developing valid statistical methods for this setting is an important open question.

A Appendix
A.1 Ridge regression as a weighting estimator Just as we can write ridge ASCM as a weighting estimator, we can write ridge regression alone as a weighting estimator.
Lemma 4. Withη ridge 0 andη ridge , the solution to the ridge regression of post-treatment outcomes on pre-treatment outcomes for control units, the ridge estimate can be written as a weighting estimator:Ŷ ridge 1 We can also characterize the imbalance and L 2 norm of the ridge regression weights: Lemma 5. Let σ 2 min be the minimum eigenvalue of the sample covariance matrix 1 N 0 X 0· X 0· . Then with κ = λ N 0 σ 2 min + 1 > 1, the ridge weightsγ ridge satisfy: and Next, we can use Lemmas 4 and 5 with Theorem 1 to characterize the bias of ridge regression under a linear factor model. To the best of our knowledge this is a novel result.
Finally, we note that ridge regression weights are a special case of the elastic-net synthetic control estimator proposed by Doudchenko and Imbens (2017) and correspond to an L 2 penalized version of the Oaxaca-Blinder weights considered by Kline (2011). Furthermore, these weights fit into the general balance weights formulation (19) with an L 2 dispersion penalty f (γ i ) = 1 2 γ 2 i and an L 2 balance criterion h λ (x) = 1 2λ x 2 2 .
Lemma 6. The ridge weights (29) are the solution to the general balancing weights problem (19) with an L 2 balance criterion and dispersion function, i.e.γ ridge is the solution to A.2 Derivation of the general dual problem We now derive the Lagrangian dual for the general balancing weights problem (19). We can augment the optimization problem with auxiliary variables , yielding: The Lagrangian is The dual maximizes the objective q(α, β) = min γ, L(γ, , α, β) where a convex, differentiable function g has convex conjugate g * (y) ≡ sup x∈dom(g) {y x − g(x)}. By strong duality the general dual problem (20), which minimizes −q(α, β), is equivalent to the primal balancing weights problem. Given theα andβ that minimize the Lagrangian dual objective, −q(α, β), we recover the donor weights solution to (19) aŝ The convex congjugate of the entropy function is f * (x) = exp(x − 1), and the convex conjugate of the exact balancing constraint is uniformly zero. Thus, in the special case of entropy penalized SCM with exact balance (15) the specialization of the general dual problem (20) is By redefining α to be α − 1, we get the dual form (16). The convex conjugate of h ζ (x) = 1 2ζ x 2 2 is h * ζ (x) = ζ 2 x 2 2 so following the general dual problem (20) the dual problem to entropy penalized SCM is We again absorb the constant subtraction term into the intercept and redefine α to be α − 1, which gives the form of the dual problem presented in the main text (18).
A.3 With large T 0 , SCM is IPW on factor loadings with ridge penalty One limitation of Theorem 1 is that it is an upper bound on the bias. However, the perspective of SCM as approximate balancing weights and its dual view as IPW provide insight into existing results in the literature that show that without exact balance, SCM will indeed be biased even with infinite pre-treatment time periods. In particular, Ferman and Pinto (2018) show that SCM will be biased by considering the behavior of the un-penalized SCM optimization problem (3) as T 0 → ∞. Under a more general set of assumptions, Ferman and Pinto (2018) show that the SCM weights converge to the minimizer of the asymptotic MSE. Under the linear factor model (10) and our set of assumptions, this implies that the solution to the unpenalized SCM problem (3) converges toγ, whereγ = argmin Notice that the un-penalized SCM problem (3) that approximately balances the lagged outcomes converges to a penalized SCM problem that approximately balances the factor loadings, where the noise level σ 2 controls the level of penalization. The dual formulation of (40) fits a propensity score with linear odds and a ridge penalty: Even as the number of pre-treatment periods goes to infinity, the estimator is still regularized: with larger noise variance σ 2 , the dual propensity score parameters shrink more towards 0 and thus the weights shrink towards uniformity. Notice that this is exactly the same phenomenon as OLS with measurement error in the covariates. Therefore, as Ferman and Pinto (2018) also conclude, even asymptotically SCM will not balance the factor loadings and thus the SCM estimator will be biased asymptotically in T 0 , except in degenerate cases.

A.4 Auxiliary Covariates
A.4.1 Auxiliary covariates and a reweighted L 2 norm The original SCM formulation (Abadie et al., 2010(Abadie et al., , 2015 includes covariates Z i ∈ R d , possibly including (some function of) the lagged outcomes, and finds weightsγ(V ) that minimize the imbalance in the covariates measured by a V -weighted L 2 norm, for a symmetric and positive definite matrix V ∈ R d×d : min where the rows of Z c ∈ R Nc×d are the augmented covariates Z i for the control units. V is typically restricted to be diagonal. Thus, rather than balancing lagged outcomes, this version of SCM minimizes imbalance across the auxiliary covariates. Abadie et al. (2010) suggest choosing V to minimize the L 2 imbalance for lagged outcomes X, when the synthetic control is formed byγ(V ): This results in a two step procedure that searches over possible values of V , solving (42) each time and choosing the V which minimizes (43). The V matrix prioritizes balance in some variables over others. In our framework the V matrix defines the balance criterion and thus the type of regularization. To see this, consider the dual (20) to the entropy-penalized version of (42) This dual problem is instructive: the original SCM fits a penalized propensity score model using Z as covariates and with a ridge penalty, reweighted by V , where V is chosen to optimize the pre-intervention outcome imbalance obtained by the estimated propensity score. This procedure does not guarantee balance on the auxiliary covariates, only using them as a proxy for the lagged outcomes. See Kaul et al. (2018) for further discussion.

A.4.2 Balancing auxiliary covariates directly
An alternative to incorporating covariates through an outcome model (as we propose in Section 7) is to explicitly balance both the auxiliary covariates and the lagged outcomes. We separate the balance criterion into two pieces, one for the lagged outcomes X i and one for the auxiliary covariates Z i . For example, we can use separate L 2 balance criteria: where the two hyper-parameters ζ X and ζ Z control the trade off between prioritizing balance in the lagged outcomes and auxiliary covariates. The Lagrangian dual (20) estimates a propensity score using both the lagged outcomes and the covariates, with separate ridge penalties: We can follow a similar procedure in the outcome model by also separately regularizing the lagged outcomes and auxiliary covariates.

A.5 Weighted randomization tests with a single treated unit
We now consider the permutation test from a randomization-based perspective. First, we outline the necessary conditions for a valid randomization test with varying, known propensity scores (Rosenbaum, 1987;Branson and Bind, 2018;Toulis and Shaikh, 2018), then we show the uniform permutation test of Abadie et al. (2010Abadie et al. ( , 2015 differs from this ideal case. We then briefly discuss a weighted randomization with a single treated unit, which we use in the simulation studies. Following Firpo and Possebom (2017), we consider the randomization-based test for the sharp null hypothesis of no treatment effect whatsoever: where we again assume that both potential outcomes are defined for all units. Consider a test statistic, S(W, Y(0), Y(1)), where W is the vector of treatment assignment indicators, W 1 , . . . , W N , and Y(0) and Y(1) are the vectors of potential outcomes. Under the sharp null in Equation (47), For simplicity, we restrict test statistics to be non-negative, for example by taking the absolute value, which yields one-sided p-values.
For the setting with a known assignment mechanism, we draw the vector of treatment assignments W ∼ Pr(W|X), with observed assignment W obs and observed test statistic S obs = S(W obs , Y obs ). Note that this is a departure from type of uncertainty discussed in the main text. The valid (one-sided) p-value is then: where I is the indicator function and where the expectation is taken with respect to the (known) randomization distribution, Pr(W|X). It is well known that the p-value in Equation (48) is valid in any finite sample, in the sense that P(p wt ≤ α) ≤ α for every α ∈ [0, 1], so long as Pr(W|X) is indeed the correct randomization distribution; see, for example Branson and Bind (2018) and Toulis and Shaikh (2018). Conversely, taking the expectation with respect to the incorrect randomization distribution may yield an invalid p-value. In particular, the commonly used uniform permutation test calculates: where S i is the test statistic when unit i receives the treatment. This will only be valid when treatment assignment is constant across all units.
To make progress, we impose additional structure on Equation (48). For simplicity, let π i ≡ P(W i = 1) be known, independent, unit-level treatment probabilities under a Bernoulli design, W ∈ {0, 1} N . To reflect standard SCM settings, we further restrict the possible randomizations to draws with a single treated unit, N 1 = 1. Following Branson and Bind (2018), the probability that unit i receives treatment is therefore Continuing the convention that unit 1 is the observed treated unit, W 1 = 1, we can therefore re-write Equation (48) as: where π i ≡ π i j =i (1 − π j ). We can then use the duality between SCM and IPW to modify p wt to obtain a valid p-value for the SCM setting with one treated unit: whereγ i are the estimated SCM weights and where γ =γ i 1+γ i j =i 1 1+γ i .Ifγ i are, in fact, the true treatment odds, then p scm will be a valid p-value, analogous to p wt . A practical complication is that unpenalized SCM does not yield a propensity score estimate for the treated unit; we need the implied propensity score link function to have a closed form to estimateγ 1 . Thus this procedure is limited to penalized variants of SCM.

A.6 Simulation data generating process
We now describe the simulations in detail. For the Basque example, N = 17 and T 0 = 20; for the California example, N = 38 and T 0 = 18. For each, we use the Generalized Synthetic Control Method (Xu, 2017) to fit the following linear factor model, setting J = 4: We then use these estimates as the basis for simulating data.Appendix Figure C.3 shows the estimated factorsμ. We use the estimated time fixed effectsν and factorsμ and then simulate data using Equation (53), drawing: whereα andσ α are the estimated mean and standard deviation of the unit-fixed effects,Σ φ is the sample covariance of the estimated factor loadings, andσ ε is the estimated residual standard deviation. We assume a sharp null of zero treatment effect in all DGPs.
To model selection, we compute the (marginal) propensity scores as where we set θ = 1/2 unless otherwise noted and re-scale the fixed effects α to have unit variance. Finally, we restrict each simulation to have a single treated unit and therefore normalize the selection probabilities π i as in Equation (50). We also consider an alternative data generating process that specializes the linear factor model to only include unit-and time-fixed effects: We calibrate this data generating process to the Basque data by fitting the fixed effects with gsynth and drawing new unit-fixed effects from α i ∼ N (α,σ α ). We then model selection proportional to the fixed effect as above. Second, we generate data from an AR(4) model: where we set ρ = 1 3 . We model selection as proportional to the last 4 outcomes logit −1 π i = θ B Proofs Lemma 7. Under the linear factor model (10), for weights γ with i γ i = 1, the bias is Proof of Lemma 7. Collecting the factor loadings into a matrix Φ ∈ R N ×J , notice that Multiplying both sides by (µ µ) −1 µ and recalling that 1 T 0 µ µ = I, this gives that Therefore the bias is Lemma 8. For weights γ and it sub-Gaussian with scale parameter σ and independent, with probability at least 1 − 2e −8J log 5 .
Finally, note that µ T 2 ≤ The 2-norm of the weights is The imbalance of the ridge regression weights is: The 2-norm of the weights is Proof of Corollaries 1 and 2. Plugging in the results of Lemmas 2 and 5 into Theorem 1 and simplifying the expression directly yields the results.
Proof of Lemma 6. The Lagrangian dual to (33) is min α,β 1 2 Solving for α we see that Since the lagged outcomes are centered, this implies that Now solving for β we see that This implies thatβ = (X 0· X 0· + λI) −1 X 1 Finally, the weights are the ridge weightŝ Proof of Lemma 3. By replacing X with Z in Lemma 1 we find the form of the weights. The imbalance is Proof of Proposition 1. The bias of Var(τ ) is We now compute E[ν T − ν T ]:  1960 1970 1980 1990 1960 1970 1980 1990 1960 1970 1980 1990 −  (Abadie and Gardeazabal, 2003) using SCM, ridge ASCM, and ASCM with covariates. Note that ridge ASCM achieves nearly perfect balance for the pre-period outcomes. Basque analysis In addition to replication the Prop. 88 study in Abadie et al. (2010), we replicate the study on the impact of terrorism on the Basque ecnonomy in Abadie and Gardeazabal (2003). As before we estimate the effect using (1) SCM on the lagged outcomes, (2) ridge ASCM on the lagged outcomes, and (3) the covariate-adjusted SCM estimator described in Section 7. 20 Figure C.1 shows the effect estimates and their standard errors, Figure C.2a shows the imbalance in the auxiliary covariates and C.2b compares the weights for SCM and ridge ASCM. Note that in covariate balance is quite good for SCM and thus SCM and ridge ASCM are nearly identical; adjusting for covariates improves balance in the lagged outcomes by roughly 7.5%.
Machine learning methods We evaluate alternative outcome models for use in ASCM. For each DGP we consider SCM augmented with (1) a random forest, (2) a two layer feed-forward neural network, (3) matrix completion using MCPanel , (4) Lasso, and (5) CausalImpact (Brodersen et al., 2015). We compare ASCM to the pure outcome models as well as pure SCM. Figure C.5 shows the absolute bias for these methods and Appendix Figure C.6 shows the RMSE. We broadly see the same results as with ridge ASCM. In our simulations, augmenting SCM with flexible outcome models always reduces the bias, with some models improving SCM more than others. Additionally, ASCM can have much better performance than either SCM or outcome modelling alone. For instance, with such small data the neural network is badly biased in almost all simulations, but augmenting SCM with a neural network model greatly reduces the bias relative to both estimators; this is especially stark in the Basque simulation, and we see similar behavior for CausalImpact in the fixed effects simulation.
Overall we find consistently good performance across data generating processes from SCM augmented with a penalized regression model. Due to this good performance, the small number of units and high dimensional structure of typical SCM settings, and their relative simplicity, we recommend augmenting SCM with penalized regression in practice. In particular we suggest using ridge regression; among the other benefits, ridge ASCM allows the practitioner to diagnose the level of extrapolation that the outcome model adds.
Alternatives to the uniform permutation test In Appendix Figure C.8 we show the empirical distribution for the p-value under strong selection using uniform permutation, uniform permutation restricting to units with positive SCM weight, two conformal inference approaches described in Chernozhukov et al. (2017), a weighted randomization test using the true propensity scores, and a weighted randomization test using propensity scores estimated with entropy-penalized SCM (14).
In general we find that the conformal inference methods and uniform permutation are invalid. The ad-hoc solution of restricting the uniform permutation to control units with positive SCM appears to be valid but lacks power, and the weighted randomization test with the estimated propensity score appears to lack any power whatsoever. This simulation evidence suggests that valid and powerful permutation-style hypothesis testing for SCM remains an open problem.  (2) i.i.d. permutations are as described in Chernozhukov et al. (2017). (3) Estimated P-Score uses entropy penalized SCM (14) to estimate the propensity score, uses the cross validation technique described in Doudchenko and Imbens (2017) to select the hyper parameter, and computes a weighted p-value (52). (4) True P-Score Uses the true propensity score in the weighted p-value (52). (5) Uniform Permutation performs a uniform permutation test over the control units and (6) Table 1: Coverage for standard errors in the Prop. 99 simulation. Standard errors are estimated using a simple non-parametric blocked bootstrap, and the leave out one method described in Section 6.2. The bias, RMSE, and average confidence interval length are all scaled by the residual standard deviation.