{'fraction_non_alphanumeric': 0.036035709101466105, 'fraction_numerical': 0.016112643344462187, 'mean_word_length': 4.470272952853598, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Difference-in-Differences for Ordinal Outcomes: Application to the Effect of Mass Shootings on Attitudes toward Gun Control

The difference-in-differences (DID) design is widely used in observational studies to estimate the causal effect of a treatment when repeated observations over time are available. Yet, almost all existing methods assume linearity in the potential outcome (parallel trends assumption) and target the additive effect. In social science research, however, many outcomes of interest are measured on an ordinal scale. This makes the linearity assumption inappropriate because the difference between two ordinal potential outcomes is not well defined. In this paper, I propose a method to draw causal inferences for ordinal outcomes under the DID design. Unlike existing methods, the proposed method utilizes the latent variable framework to handle the non-numeric nature of the outcome, enabling identification and estimation of causal effects based on the assumption on the quantile of the latent continuous variable. The paper also proposes an equivalence-based test to assess the plausibility of the key identification assumption when additional pre-treatment periods are available. The proposed method is applied to a study estimating the causal effect of mass shootings on the public's support for gun control. I find little evidence for a uniform shift toward pro-gun control policies as found in the previous study, but find that the effect is concentrated on left-leaning respondents who experienced the shooting for the first time in more than a decade.


Introduction
The difference-in-differences (DID) design is widely used in observational studies with repeated observations over time (Card and Krueger, 1994;Angrist and Pischke, 2008;Lechner et al., 2011). It allows scholars to identify the causal effect accounting for time-invariant unobserved confounders. Although significant progress has been made to improve the original DID design in recent years (e.g., Abadie, 2005;Athey and Imbens, 2006;Qin and Zhang, 2008;Lee, 2016;Arkhangelsky et al., 2018;Callaway and Sant'Anna, 2018;Li, 2019;Lu, Nie and Wager, 2019), most of the existing methods attempt to identify and estimate the treatment effect under the linearity assumption (Abadie, 2005). This parallel-trends assumption imposes a restriction on the potential outcomes such that the mean of the treatment and the control group has identical trends in the absence of the treatment. Therefore, the assumption is meaningful only when the difference between two potential outcomes is well defined (e.g., continous outcomes).
In social science research, however, many outcomes of interest are measured on an ordinal scale. For example, in political science, scholars measure voter's ideology on a scale from "very liberal" to "very conservative" (e.g., Gay, 2002;Jessee, 2016;Mason, 2015) or ask an attitude toward a policy item from "strongly disagree" to "strongly agree" (e.g., Grose, Malhotra and Van Houweling, 2015;Frymer and Grumbach, 2020;Likert, 1932). In fact, due to the limitation of space and other administrative reasons, most of the questions asked in major social science surveys are ordinal. When the outcome is measured on such a scale, it is difficult to define the "mean" of non-numeric variables and further impose a restriction on their "differences." In addition, the usual definition of the treatment effect as the difference between two potential outcomes is not well defined (e.g., Volfovsky, Airoldi and Rubin, 2015; Lu, Ding and Dasgupta, 2018), unless strong assumptions about the scale are imposed. This implies that the standard DID cannot be directly used for ordinal outcomes.
With a dearth of methods tailored for analyzing ordinal outcomes in the DID setting, scholars often treat them as continuous, dichotomize them with some threshold, or employ the ordered logistic (probit) regression. Each of the three approaches has its own shortcomings. By treating the ordinal outcome as continuous, scholars implicitly assume that categories are equally spaced. This assumption is not testable nor appropriate in many applications.
Although dichotomizing the outcome appears to enable scholars to adopt the standard DID method to estimate casual effects, this strategy is not robust to different transformations (i.e., different choices of the dichotomization threshold). Specifically, due to the non-linear nature of the ordinal outcome, the validity of the parallel trends assumption under one trans-formation does not guarantee the validity of the assumption under another transformation. This is problematic because oftentimes scholars do not have substantive justification on which transformation should be employed.
In this paper, I develop a methodology for estimating causal effects for ordinal outcomes with repeated observations. Instead of assuming linearity on the actual outcome, I utilize the latent variable formation often used in the discrete choice models. Because the assumptions are imposed on the entire distribution of the latent variables, the proposed method does not require a transformation of the outcome variable. Furthermore, the method enables researchers to estimate interpretable causal effects, defined as a difference between two probabilities, under a single set of assumptions. I also propose a diagnostic tool when scholars have data from more than one pre-treatment period. As in the standard DID for continuous outcomes, where scholars can check if the pre-treatment trends are parallel, this diagnostic test allows researchers to formally confirm whether the assumption holds at least during the pre-treatment periods.
The method of this paper is closely connected to the literature on non-linear DID (e.g., Athey and Imbens, 2006;Sofer et al., 2016;Callaway, Li and Oka, 2018;Glynn and Ichino, 2019). In particular, Athey and Imbens (2006) consider an extension of their method to binary and count outcomes, but they do not consider the case of ordinal outcomes. Most importantly, because they impose minimal restrictions on the potential outcome, their method does not provide point identification even for an additive effect and the bound can be uninformative.
Instead, I impose a stronger assumption for the sake of identifying the non-additive causal effect, which enables researches to estimate informative causal effects.
The proposed methodology is used to revisit a recent debate on the effect of mass shootings on attitudes towards gun control regulations (Barney and Schaffner, 2019;Hartman and Newman, 2019;Newman and Hartman, 2019). In their original and follow-up studies, Hartman and Newman argue that a mass shooting increases people's support for stricter gun control policies regardless of their partisanship. They also argue that the effect is conditional on geographical context, such as the safety of their neighborhood. On the other hand, Barney and Schaffner argue that there is no strong evidence to support the claim of Hartman and Newman. They also find a polarizing effect of mass shootings where Democrats become more supportive of gun control while Republicans become less supportive of gun control.
In Section 4, I re-analyze the data from the motivating empirical study using the proposed method. Using two-wave panel data, I find that mass shootings have an effect on those who experience mass shootings for the first time in a decade: they form a stronger opinion that supports gun control regulations, while the result suggests little evidence for a uniform shift toward pro-gun control policies. I also find that the effect is concentrated among Democrats including those who weakly identify themselves as Democrat. However, the effects among Republicans are not statistically distinguishable from zero. Thus, I find little evidence to support the polarizing effect of mass shootings. Using three-wave panel data, which provides an additional pre-treatment time period, I assess the plausibility of the identification assumption.
The proposed testing procedure finds a supportive evidence for the validity of the assumption.
Reanalysis of the three-wave panel, however, finds little evidence to support the claim that mass shootings have any effect on the support for gun control.
The rest of the paper is organized as follows. Section 2 introduces the motivating application of the method. In Section 3, the methodology is introduced where I discuss identification assumptions and estimation strategy. In Section 4, I apply the proposed method to the data described in Section 2. Finally, I offer concluding remarks in Section 5.  (Barney and Schaffner, 2019;Hartman and Newman, 2019) study the relationship between experiencing mass shootings and the attitude to gun control. These studies use survey data with a two-wave and three-wave panel to investigate whether living in close proximity to mass public shootings has a causal impact on people's attitude to gun control. Respondents to the survey are considered as "treated" if at least one mass shooting occurs within 100 miles from their residential zip code. To measure the attitude toward gun control, the authors used a response to the following survey question in the Cooperative Congressional Election Study (CCES) (Kuriwaki, 2018;Schaffner and Ansolabehere, 2015): In general, do you feel that laws covering the sale of firearms should be made more strict, less strict, or kept as they are?
The original studies utilize variations in treatment assignment over time to isolate the effect of mass shootings from the time trends and location effects. Based on the analysis, Hartman and Newman find that living in near proximity to mass public shootings moves people to support stricter regulations on gun sales (Newman and Hartman, 2019;Hartman and Newman, 2019). They also report that the effect does not vary by respondents' party affiliation. In a follow-up study, Barney and Schaffner (2019) correct data and conduct additional analyses. They conclude that the effect varies by which party people affiliate with and that there is a polarizing effect of mass shootings. Democrats become more supportive of stricter gun control, while Republican become less supportive of regulations.
Throughout the debate, the authors utilize a variety of methodologies, such as a ordered probit model with random effects and a linear fixed effect model, to estimate the impact of mass shootings on the attitude (see Table 1 in Appendix E). Although "difference-indifferences" is mentioned in these studies, discussions about the quantities of interest and assumptions required for the identification of those quantities are missing from the debate.
Without assessing the assumptions explicitly, it is challenging to conclude whether mass shootings have any effect on people's attitude. This paper fills this gap by proposing a methodology that enables researches to assess assumptions and reliably estimate causal effects.

The methodological challenge
For a causal effect to be reliably estimated, assumptions must be imposed and evaluated. In the following, I demonstrate that even if researchers are aware of identification assumptions, the current practice is not well suited for the ordinal outcome. Suppose that, following common practice, researchers dichotomize an ordinal outcome into a binary outcome by specifying some threshold. The standard DID analysis is then applied on this transformed outcome. In our running example, there are two possible ways to transform the original outcome into a binary variable. One way is to code more-strict category as one and kept-as-they-are and less-strict as zero; the other way is to treat more-strict and kept-as-they-are as one and less-strict as zero.
After transforming the outcome, scholars can check if the pre-treatment trends are parallel for this new binary variable. Inspecting pre-treatment trends is a routine often used in empirical studies to justify the use of the DID design (Angrist and Pischke, 2008). Figure 1 shows trends for each transformed outcome using the three-wave panel of the survey where I subset respondents who are not treated until 2012. The panel on the left shows the first type of transformation where only more-strict category is coded as 1 (denoted by Y on the y-axis). We can see that the pre-treatment trends between the treatment group (blue)  Figure 1: Visual assessment of parallel trends assumption for the three-wave panel from the survey data. Respondents who are not treated until 2012 are used for generating this plot. The lines with solid circles (blue) show trends for the treated group and the lines with hollow circles (gray) show trends for the control group. Vertical dashed lines (red) show the timing of the treatment. Although the left panel appears to show that pre-treatment trends are parallel between the treatment and the control group, the right panel suggests that pre-treatment trends are not parallel. and the control group (gray) appears to be parallel (denoted by D on the y-axis.). Thus if a researcher transforms the original variable in this way, she might conclude that the DID design is suitable for analyzing the data. The panel on the right shows the pre-treatment trends for the second type of transformation. In this case, however, the pre-treatment trends do not seem to be parallel: the trends cross during the pre-treatment period.
Note that this observation is not specific to this application. In Appendix C, I demonstrate that it is trivial to construct an example that satisfying parallel trends in one transformation does not imply the parallel trends in another transformation.
It is often unclear ex ante which threshold should be chosen from a substantive point of view. Therefore, it is unfortunate that the validity of the design appearently depends on how the variable is transformed. Although the running example only has three categories, the problem exacerbates when scholars need to analyze an outcome that has a larger number of categories.

The setup
Let Y it ∈ {0, . . . , J − 1} ≡ J denote the observed outcome measured on an ordinal scale with J categories (J ≥ 3) for unit i ∈ {1, . . . , n} and time t ∈ {0, 1}. The binary treatment, denoted by D i ∈ {0, 1}, is assigned after Y i0 is observed but before time t = 1. We use the potential outcome notation to denote the counterfactual outcome, For example, Y i1 (0) is an attitude toward gun control regulations that would realize in the post-period if a respondent did not experience a mass shooting (i.e., the control condition).
In many applications, scholars are interested in estimating the distributional treatment effect. In this paper, I focus on the treatment effect on the treated. Specifically, the effect ζ j is defined as the difference in probabilities of choosing category j under two conditions, for j ∈ J . In our application, ζ 2 is the difference in probabilities that those treated prefer more strict gun control between the treated and the control conditions. Thus, observing ζ 2 > 0 implies that the mass shootings make people prefer stricter policies on gun control.
Similarly, ζ 0 is the effect of the treatment on less-strict category and ζ 0 > 0 implies that incidents turn people to prefer less strict regulations.
When the number of categories is large, it is sometimes useful to estimate the cumulative effect ∆ j , which is defined as a difference in probabilities of choosing j or larger categories under the two conditions, for j ∈ J \{0}. Note that ∆ j = J−1 =j ζ by construction, and thus it is sufficient to consider the identification of ζ j .
The cumulative effect is also useful to connect the approach that dichotomizes ordinal outcomes to the proposed method. From the above definition, we can see that the standard DID based on the dichotomized outcome at threshold j identifies ∆ j . This is because is the dichotomized potential outcome with threshold j. This means that the standard DID applied to the dichotomized outcome can estimate only one of J − 1 possible quantities of interest. Furthermore, if one wishes to estimate all possible ∆ j 's by changing the threshold, it requires J − 1 distinct identification assumptions. As we saw in our motivating example, however, satisfying the parallel trends assumption for ∆ j does not necessarily imply that the assumption for ∆ j is satisfied.

Identification
Typically, we do not have a good sense of which estimand is best suited for answering the substantive question. Therefore, it is natural that we attempt to identify and estimate ζ j for all j ∈ J from the observed data. The goal of this section is to establish the identification of ζ = (ζ 0 , . . . , ζ J−1 ) with a single set of assumptions.
To compute the quantity defined in Equation (3.1), we need the marginal distribution (0), we need to impose additional assumptions to identify the distribution of Y i1 (0) for D i = 1. Following Athey and Imbens (2006), I omit the subscript i for units and Y dt denotes the potential outcome under the control condition at time t for group defined by is what we do not observe in the data. In our example, Y 11 is the potential attitude to gun control that we would have observed if those respondents who have experienced mass shootings would have not been exposed to the event.
I first impose a structure on the potential outcome. Specifically, I assume that the observed categorical outcome follows the index model, which means that there is a latent variable behind Y dt and that the categorical outcome is defined by a simple thresholding rule on the latent variable.
Assumption 1 (Index model). Assume that the potential outcomes follow the index model such that there exists a latent variable Y * dt ∈ R and where {κ j } J j=0 are a set of cutoffs with κ 0 = −∞ and κ J = ∞.
Assumption 1 says that the potential outcome defined on an ordinal scale Y dt is a function of another potential outcome defined on a continuous space Y * dt . In the application, Y * dt can be considered as the underlying intensity of one's attitude toward gun control policies where larger value of Y * dt corresponds to a support for stricter gun control. The assumption allows us to handle the outcome on a continuous space through Y * dt instead of directly working on a discrete space. Note that κ j 's are constants assumed to be fixed and they do not depend on group (d) nor time (t).
Different from the additive effect, the distributional treatment effect ζ j requires that the entire marginal distribution of the potential outcome is identified. For that, I further impose a distributional assumption on Y * dt in Assumption 2.
Assumption 2 (Location-scale family assumption). Let U denote a continuously distributed random variable with mean 0 and variance 1 that belongs to a parametric family. We assume that Y * dt belongs to the location-scale family, that is, it can be written as where µ dt is the location and σ dt is the scale parameter.
Assumption 2 specifies the distribution of the latent utilities. It assumes that each marginal distribution belongs to the location-scale family distribution with time and group specific location and scale parameter. This implies that the distribution of the potential outcomes are different up to mean and the scale. Note that the joint distribution of the latent utilities are left unspecified, so units can have correlated latent utilities over time. Although this is a parametric assumption (i.e., the distribution of U should be known), the location-scale family encompasses a large class of parametric distributions (e.g., the normal distribution, the logistic distribution or the t-distribution, etc).
Finally, I impose a structure on the relationship between latent variables Y * dt . This allows us to map what we observe in the control group over time to what would have happened to the treated group if it was not treated. I first start with a restrictive assumption that is similar to the standard DID design. It is possible to assume that the parallel trends hold on the latent outcome, that is, Then, the mean of the counterfactual latent outcome Y * 11 is uniquely identified as µ 11 = µ 10 + µ 01 − µ 00 .
However, this approach is restrictive, because it requires an additional assumption that the variance is constant across time and groups, that is, σ dt = σ for all d and t; otherwise we cannot identify the entire distribution of the latent outcome Y * 11 . This constant variance assumption is strong because it only allows the unidirectional change of choice probabilities. Therefore, I impose a different assumption from the standard parallel trends assumption.
Instead assuming the mean shift, the assumption is imposed on the entire distributions, which is originally introduced by Athey and Imbens (2006) (also see Sofer et al., 2016). Specifically, I assume that the shift in the distribution across time are constant between the treatment and the control groups. Figure 2 graphically illustrates the assumption. The key part of this assumption is that the vertical arrows in the two graphs should be the same length. In other words, q d (v) − v captures the trend in the distribution (i.e., how much Y * dt "shifts" between t = 0 and t = 1) and the assumption says that the "shift" is identical across two groups. This means that for each choice of v, the corresponding value of q d (v) ("shift") should be the same for d = 0, 1.

Assumption 3 formally states the assumption.
Assumption 3 (Distributional parallel trends (Athey and Imbens, 2006) Assumption 3 imposes a restriction on the relationship between the pre-treatment latent outcome Y * 10 and the counterfactual latent outcome Y * 11 , based on the relationship between two latent variables in the control group. Note that by construction The key assumption is that the length of the vertical arrow (red) is the same between the two panels for all range of v. This allows us to recover the shape of the dashed line based on latent utility distributions for the observed outcomes (i.e., solid lines). and 3, the distribution of the counterfactual latent utility Y * 11 is identified as where µ 11 = µ 10 + µ 01 − µ 00 σ 00 /σ 10 and σ 11 = σ 10 σ 01 σ 00 .
And thus, the distribution of the potential outcome is identified as A proof is in Appendix A. Proposition 1 says that the location and the scale of the counterfactual latent outcome Y * 11 are uniquely determined by parameters of observed outcomes. This implies that we can recover the distribution of the counterfactual outcome Y 11 ∼ Y i1 (0)|D i = 1 (i.e., the potential outcome under the control condition for the treated unit at time t = 1) using parameters estimated from the observed data, Y 00 , Y 01 and Y 10 . For example, if we assume that U follows the standard normal distribution, we have that Y * 11 follows the normal distribution with mean µ 11 and variance σ 2 11 .

Estimation
The identification result in the previous section provides a guidance on how we can estimate the causal effect from the observed data. Let θ dt = (µ dt , σ dt ) denote a vector of parameters that characterize the distribution of the latent utility Y * dt . I take a two-step approach to estimate causal quantity ζ j defined in Equation 3.1 for all j ∈ J . In the first step, I estimate parameters for observed outcomes, that is, θ 00 , θ 01 and θ 10 . Based on the estimate of these parameters, causal effects are estimated in the second step.
In this and the following section, I focus on a case where Y * dt follows the normal distribution, that is U ∼ N (0, 1). Then, by Assumption 2, the observed outcomes Y 00 , Y 01 and Y 10 follow the ordered probit model. Thus, parameters θ = (θ 00 , θ 01 , θ 10 , κ ) can be estimated via the maximum likelihood.
where 1{·} is an indicator function that takes 1 if the argument is true and takes 0 otherwise, and Φ(·) is the cumulative distribution function of the standard normal distribution. Different from the standard ordered probit specification, I fix two cutoffs κ 1 and κ 2 (recall that κ 0 = −∞ and κ J = ∞). This allows us to estimate the variance component in addition to means (Lemma 1 in Appendix A; also see for example Jackman, 2009, Chapter 8). Note that the choice of κ is not consequential in that, the causal effect estimate ζ is invariant to the choice of the cutoffs (Lemma 3 in Appendix A). This is because the identification assumption imposes a structure on the quantile scale, which is invariant to the scale of the latent variables, while different choices of cutoffs only affect the location and the scale (i.e., µ and σ) of the latent variables.
We then estimate the parameter for the counterfactual distribution θ 11 = (µ 11 , σ 11 ) by the plug-in estimator based on the first stage, µ 11 = µ 10 + ( µ 01 − µ 00 )/( σ 00 / σ 10 ), and σ 11 = ( σ 10 σ 01 )/ σ 00 . (3.8) Since the causal effect is a function of θ 11 , the estimator for the causal effect is therefore given by where n 1 = n i=1 D i , and then ∆ j = J−1 =j ζ j . Note that the first term of the right-hand side is a nonparametric estimator of P(Y i1 (1) = j | D i = 1) because this quantity is identified from the data without any assumptions. The second term on the right-hand side is the counterfactual distribution identified by the assumptions (Proposition 1).
Lemma 4 in Appendix A establishes the √ n consistency of the estimator ζ, whose sampling variance can be derived using the delta method under the independence assumption. In practice, however, the block bootstrap can be used to estimate the variance when outcomes are correlated across time or due to clustering.

Assessing the distributional parallel trends assumption
In the standard DID design, additional pre-treatment periods provide an opportunity to assess the parallel trends assumption by checking the pre-treatment trends (Angrist and Pischke, 2008;Egami and Yamauchi, 2019). Although it is not a direct test of the assumption, observing the parallel trends in the pre-treatment periods suggests that the assumption is more likely to be plausible. With a similar logic, we can assess the validity of distributional parallel trends assumption (Assumption 3). Specifically, we would expect that if the distributional parallel trends holds for the pre-treatment periods, it is more reasonable to claim that the assumption holds in the post-period. Thus, we assess the validity of Assumption 3 by testing if the similar condition holds for the pre-treatment periods.
The proposed testing procedure Suppose that we now observe the outcome for three where Y i2 is the post-treatment outcome and Y i0 and Y i1 are the pre-treatment outcomes. The treatment is administered after time t = 1 in this setup, and thus we have Y it (0) = Y obs it for t = 0, 1 regardless of the treatment status. This means that observed outcome before the treatment assignment is the same as the potential outcome under the control condition for both treatment and control groups.
The assumption requires that two functions are identical on the unit interval, that is, q 1 (v) = q 0 (v) for all v. Therefore, we wish to statistically test ifq 1 (v) =q 0 (v) holds for all v ∈ [0, 1] using the data from the pre-treatment periods.
Intuitively, we can check the equivalence of two functionsq 1 andq 0 by assessing the maximum deviation between two functions, If this metric is "small", we may conclude thatq 1 =q 0 . Formally, with some threshold δ > 0, we wish to test the following hypotheses: where H 0 says that two functions are not equivalent (i.e., large deviation). Rejecting the null implies that the data supports H 1 of equivalence which is what we want to demonstrate.
For now, I assume that researchers know how to choose an appropriate value of δ based on substantive knowledge. I will discuss how to calibrate this equivalence threshold in the below.
We can see that the null hypothesis can be written as a union of two hypotheses without absolute values, This decomposition implies that we can conduct two one-sided tests to determine if we reject the original null H 0 or not. In other words, we conclude that H 0 is false if we reject both The detail of how to construct the confidence interval is presented in Lemma 6 and 7 in Appendix A. Then, by the one-to-one relationship between the test and the confidence set, we reject H + 0 if and only if the upper confidence interval is less than δ, that is By the similar argument, we reject Proposition 2 shows that the proposed procedure is in fact asymptotically level α test, that is, it rejects the null of non-equivalence with probability less than α when the null is true.
Proposition 2 (Validity of the Testing Procedure). For a given choice of the equivalence threshold δ and the level of a test α, the testing procedure asymptotically controls the type I error, that is, under the null H 0 : t max ≥ δ, The above proposition shows that when the equivalence threshold is chosen such that the null is true (i.e., t max ≥ δ), then the probably to falsely reject the null (type I error) is less than α, for any value of t that is consistent with the null. In other word, the proposed testing procedure is statistically valid for any choice of the equivalence threshold under the null.
The above result suggests that we can also compute the p-value for this test by solving the rejection rule with respect to α, Intuitively, the p-value for the test is the maximum of all point-wise p-values because we are testing the maximum deviation ofq 1 (v) −q 0 (v).
Choosing an equivalence threshold δ So far, we have assumed that researchers have a clear idea what value should be used to assess the equivalence. When researcher have substantive knowledge about the appropriate value of δ given an application, it is reasonable to choose δ according to the knowledge. Oftentimes, this approach might not be feasible since it is not straightforward to form an idea of what value of δ should be deemed appropriate, especially when the value of δ is not directly tied to interpretable quantities such as causal effects. Although, any choice of δ is a valid choice because type I error is controlled for the corresponding null hypothesis, it seems useful to suggest a reasonable default value of δ to facilitate the practical use of the method.
This is a threshold used in the conventional KS test which is a nonparametric test on the difference between two distribution functions. The test is based on the maximum difference between two cumulative distribution functions. In KS test, the value of ω by the level of a test, but it is fixed here. The key feature of this threshold is that δ n depends on the sample size.
The equivalence threshold that depends on the sample size is discussed in Romano (2005).
Intuitively, this selection of δ implies that we raise the standard of what the equivalence means as the sample size increases. Therefore, rejecting the null with larger n will be a stronger evidence for the identification assumption.

Empirical Findings
In this section, I revisit the empirical application introduced in Section 2. We first reanalyze the two-wave panel of CCES (2010CCES ( -2012. This two-wave panel is the data used for main analyses in the original studies. We then analyze the three-wave panel of CCES (2010-2012-2014) which allows us to assess the identification assumption using the pre-treatment periods.
In the following, we focus on estimating the following causal quantities: for j = 0, 1, 2. Recall that less-strict is coded as 0, and more-strict category is coded as 2.

Result from the two-wave panel
This section presents a result of the analysis on the two-wave sample from CCES (n = 16620).   We can see that causal effect estimates are not statistically significant at the 10% level for all categories in the full sample. Estimates are precisely estimated and they are all close to zero, indicating that there is little evidence to suggest that mass shootings have, on average, any effect on the attitude towards gun control regulations among those who live in their vicinity of public shootings.
Following the original authors, I conduct two sets of subgroup analysis by "prior exposure" status and by the partisanship. The analysis reveals a similar pattern that most of the estimates are not statistically distinguishable from zero at the conventional level. However, we can also see that heterogeneity exists: the "no prior exposure" group has negative effect for the middle category ( ζ 1 = −0.035, SE = 0.016) which is statistically significant at the 5% level. This result implies that those living in the safer neighborhood (i.e., "no prior exposure") move away from the status quo. Although not statistically significant at the 10% level, we can also see that the effect on the less strict category is positively estimated for this "no prior exposure" group, indicating that the shift away from the status quo was probably not uni-directional. We can also see the negative effect on less-strict category among the independents ( ζ 0 = −0.026, SE = 0.011), which is statistically distinguishable from zero at the 5% level. In Appendix D, I present a result using a more granular measure of partisanship used in the survey, which asks respondents to categorize themselves on a 7-point scale from "strong Democrat" to "Strong Republican." The result in Figure D.1 shows that the effect is concentrated among lean Democrats, 91% of them categorized themselves as "independent" on the 3-point partisanship scale.
To further investigate the interactive effects between the the partisanship and prior exposure status, I considered interactions between the two variables. Figure 4 shows the results of the analysis. As we can see the effect is concentrated among Democrats who are in the "no prior exposure" group, while none of the effects are statistically significant for other partisans.
The figure also shows that partisanship does not play a role in the "prior exposure" group where estimates are indistinguishable from zero.
Finally, Barney and Schaffner (2019) consider different thresholds to determine who are "exposed" to the mass shootings. In addition to the above 100 mile threshold, I also estimate effects for the 25 mile threshold. The result is shown in Figure D Figure 4: Estimated treatment effects with 90% (solid) and 95% (thick) confidence intervals. Circles indicate effects for less-strict (ζ 0 ), squares for keep-as-they-are (ζ 1 ) and triangles for more-strict (ζ 2 ). Text labels above the estimates indicate subsamples used for the analysis.
are statistically significant at the 5% level. I apply the diagnostic test proposed in Section 3.4 to the pre-treatment outcome. The goal here is to statistically test if the condition of the distributional parallel trend holds, namely,

Concluding Remarks
In spite of the recent developments in the literature on the DID design, less attention has been paid for when the outcome is measured on an ordinal scale. In this paper, I proposed a method that allows scholars to leverage ordinal outcomes without making the linearity assumption as in the the standard DID analysis. I also proposed a procedure that assesses if the key identification assumption is plausible when additional pre-treatment periods are available.
This enable scholars to inspect the data and to discuss if the assumption is reasonable given a particular dataset they analyze, which is a crucial step for any research that attempts to establish a causal relationship.
Several extensions of the proposed methods are possible. In Appendix B, I demonstrate that the proposed method can be useful to estimate other types of causal estimands such as the proportion of who benefits from the treatment (e.g. Lu, Ding and Dasgupta, 2018).
Recent years, it has been argued that such estimands are preferable because the distributional treatment effects considered in the main text are not necessarily easy to interpret. Because the proposed method identifies the entire distributions of the potential outcomes, it is possible to compute any causal estimand that is a function of marginal distributions of the potential outcome. In the appendix, I also discuss how to incorporate time-varying covariates, which requires a further modeling assumption.
In Appendix F, I present a simulation study where I assess the finite sample property of the proposed estimator and the testing procedure. I find that under the correct model specification, the proposed method outperforms the two competing methods: the standard difference-in-differences on the dichotomized outcome and the ordered probit model. I also find that the type-I error is properly controlled for the proposed testing procedure, while the power depends on the choice of the equivalence threshold.

A.1 Lemmas
Before proving propositions, we present useful lemmas.
Lemma 1 (Identification of mean and variance of the latent variables.). Suppose that the cutoffs are fixed at κ 1 and κ 2 for Y dt = j ∈ {0, 1, 2}. Then, µ dt and σ dt in Y * dt ∼ µ dt + σ dt U are uniquely identified from the observed probability distribution.
Proof of Lemma 1. Suppose that U has the density f U (u). Then, we can form a non-linear system of equations which are sufficient for estimating µ and σ.
Lemma 2 (Alternative formula for identification). Suppose Y dt = j ∈ {0, 1, 2}. Let v 1 = F 01 (κ 1 ) and v 2 = F 01 (κ 2 ) where κ is a set of fixed cutoffs. Under Assumption 1, 2 and 3, we identify µ 11 and σ 11 by the following system of non-linear equations: Proof of Lemma 2. Under the distributional parallel trends assumption, we have q 0 (v) = q 1 (v) for all v ∈ [0, 1]. Then, where the first equality is due to the definition of q d (v) and the last equality follows by Assumption 2. Pick v 1 and v 2 as in the statement. Drawing on a similar to the argument in Lemma 1, we obtain the identification.
Proof of Lemma 3. Let F dt (y) = Pr(Y * dt ≤ y) denote the cumulative distribution function of the latent variable Y * dt under the cutoff κ. Similarly, let F dt (y) denote the CDF under κ . To show the causal effect estimates are invariant to the choice of cutoffs, it is sufficient to show that F 11 (y) = F 11 (y), that is, the invariance of the identified counterfactual latent distribution.
Applying the result of Lemma 2, we can see that µ 11 and σ 11 are uniquely identified under different sets of cutoffs, that is, ζ(κ) = ζ(κ ).
Lemma 4 (Asymptotic Normality of Causal Estimates). Under some regularity conditions, as n → ∞ with n 1 /n → k, we have that Proof of Lemma 4. We prove the statement by showing the following two statements: 2. θ 11 = ( µ 11 , σ 11 ) is √ n-consistent estimator for θ 11 .
We then use the continuous mapping theorem for the convergence in distribution to obtain the final result.
To be clear, we (sometime implicitly) condition on D i = 1 throughout the proof, which assumes that there is a super population of units with (1) = j} | D i = 1] = π 11 (j) as n → ∞ with n 1 /n → k by the law of large numbers. This proves the consistency. By the central limit theorem, it also follows that N 0, Var( π 11 (j)) k 2 as n → ∞ with n 1 /n → k. Next, recall that θ 11 is given as a transformation of θ 00 , θ 01 and θ 10 , all of which are MLE of the problem described in Section 3.3. Therefore, under the assumption of correct model specification, we obtain that θ 00 , θ 01 and θ 10 are jontly asymptotically normal centered around θ 00 , θ 01 and θ 10 . By the continuous mapping theorem, it follows that θ 11 is also asymptotically normally distributed.
Finally, using the continuous mapping theorem again, we have that ζ j is a √ n consistent estimator for ζ j where the variance σ j is obtained by the delta method (See Lemma 6).
Lemma 5 (Asymptotic Normality of θ for Pre-treatment Parameters). Let θ = (θ 00 , θ 01 , θ 10 , θ 11 ) , all of which are estimated using the data from the pre-treatment periods. Then, under regularity conditions in Newey and McFadden (1994), the maximum likelihood estimator θ is asymptotically normal with covariance Ω, where Ω is a block-diagonal matrix under independence assumption.
Proof of Lemma 5. The result is a direct application of the standard result of the maximum likelihood estimation. Therefore, proof is omitted.
Lemma 6 (Asymptotic Distribution of the Test Statistic). Assume that U ∼ N (0, 1). Let where θ is evaluated at the truth, Ω is the asymptotic variance covariance matrix of θ given in Lemma 5, and the gradient takes the form of Proof of Lemma 6. Recall that the derivative of the error function is given by which is differentiable with respect to z. Now, I compute the derivative of t(v; θ) with respect to θ, Given that Lemma 5 establishes the asymptotic normality of θ, the result immediately follows by the application of the Delta method. Then, we get From here, we obtain the variance formula as Lemma 7 (Validity of (1 − α) level sets (Liu et al., 2009)). Let t(v) =q 1 (v) −q 0 (v). Suppose U 1−α (v) and L 1−α (v) are point-wise upper and lower (1 − α) level confidence intervals, respectively such that Proof of Lemma 7. Recall that U 1−α (v) is a point-wise 100(1 − α)% level confidence interval.
This implies that . Then, we have that
Combined with the fact that U is a continuous and parametric distribution, we recovers the distribution of Y * 11 .
Proof of Proposition 2. Case 1: t ≥ . In this case, the test makes a "mistake" because the upper bound does not cover . Thus, we can focus on an event Now, consider a particular value of v. Then, we have that where the last inequality uses the fact that asymptotically U 1−α is a (1 − α) level confidence interval (Lemma 7). Case 2: t ≤ − . In this case, we focus on the other event where the last inequality is a direct application of Lemma 7.

B.1 Other estimands
This section provides an extension of the proposed methodology. Following the recent developments in the literature on causal inference with ordinal outcome, where more interpretable estimands have been proposed, I show how to apply the proposed method to those new estimands.
In the above section, we have focused on particular causal estimands, ζ j and ∆ j , which is a difference in probabilities defined for a specific reference category j. One issue of this quantity ∆ j is that depending on the choice of reference category j, the sign of the estimate might flip. This means that interpretation becomes tricky because it is completely possible to observe ∆ j > 0 and ∆ j < 0 for j = j with the same data. From this, we cannot even conclude that the treatment had "positive" effect or not.
To circumvent this problem associated with ∆ j , recent papers turn to different kinds of estimands for ordinal outcome (e.g., Volfovsky, Airoldi and Rubin, 2015; Chiba, 2017;Lu, Ding and Dasgupta, 2018;Lu, 2018). For example, Lu, Ding and Dasgupta (2018) considers the following estimand: This is a proportion units of who benefit from (or at least not harmed by) the treatment. In our example, η captures the proportion of treated respondents who change their opinion toward gun control (regardless of their baseline attitudes) after experiencing the mass shooting in their neighborhood. This quantity is easy to interpret because it does not depend on the baseline attitude and smaller value of η indicates that there are few respondents who change their opinion towards gun controls. However, since η depends on the joint distribution of potential outcomes, (Y i1 (0), Y i1 (1)), it cannot be point identified. Lu, Ding and Dasgupta (2018) provides a closed form bound for this estimand using only the marginal distribution of Y i1 (1) and Y i1 (0). The benefit of the proposed method over the dichotomizing-the-outcome approach is that it identifies the entire distribution of Y i1 (0) | D i = 1, whereas the information about the entire distribution is lost when we use the coarsened outcome. This implies that we can estimate the bound based on θ 11 given in Equation (3.8). Following the result of Lu, Ding and Dasgupta (2018), we can estimate the bound [ η, η] as where ∆ j is the estimate of the distributional effect, and ∆ 0 = 0 by construction. We can see from the formula that the upper bound is informative as long as there is at least one reference category j such that ∆ j < 0 for j = 1, . . . , J − 1. Otherwise, ∆ 0 = 0 will be the minimum and thus we get the non-informative upper bound η = 1.

B.2 Time-varying covariates
Researchers might want to incorporate time-varying covariates into the analysis to further gain efficiency. I discuss that the parametric specification of the proposed method allows the use of such covariates for analysis. Although sometimes appealing, I emphasize that parametric specification introduces additional assumptions for the analysis. Let X it ∈ R p denote a p dimensional vector of time varying covariates. We can model the mean and the variance of the latent utilities as . Then, we can express the observed choice probability as We estimate parameters γ = (γ 0 , γ 1 ) by the maximum likelihood.
Finally, the quantities of interest is estimated by taking the sample average of predicted probabilities.
Note that the marginalization of covariates is with respect to the distribution for the treated, because our estimand ∆ j is defined for the treated units.

C Dichotomizing the Outcome: An Example
Coarsening the ordinal outcome into a binary variable is a common practice often employed in applied works. Although this procedure allows scholars to utilize the standard linear DID, I will demonstrate in this section that this operation leads to an inconsistent result depending on how the new variable is created. To see this, let's consider a simple example with three categories, Y it ∈ {0, 1, 2}. There are two possible ways to transform this variable into a binary outcome, Under this setup, we require two separate parallel trends assumptions for identification, be the conditional probability for the potential outcome under the control. Now consider the following data generating process which specifies the marginal distributions for the potential outcome: However, PT2 does not hold because Thus, this example demonstrate that with the same data, ∆ 2 can be consistently estimated with Y it but ∆ 1 cannot be estimated without bias, even though we have the same data generating process behind the two transformations. Obviously, it is also trivial to construct an example where PT2 holds but PT1 does not.

D.1 Different coding of partisanship
Given the partisan nature of the gun control policies, it is important to understand if the effect of mass shootings could differ by respondents' party identification. The coding of partisanship, however, slightly different across studies. In the main text, I relied on the 3-point scale party identification question (pid3), which asks respondents the following question, Generally speaking, do you think of yourself as a ...?
In the survey, respondents are also asked to place themselves on a granular scale of partisanship (pid7):  We can see that the effect is concentrated among "Lean Democrats." On the other hand, Barney and Schaffner (2019) constructs the partisanship variable based on pid7 but collapses it into a 3-point scale, "Democrat", "Independent" and "Republican". The major difference from the self-reported pid3 is that "leaners" are classified as partisans (i.e., not independents). Figure D.2 reports the estimate based on the partisan coding of Barney and Schaffner (2019).

D.2 Different estimands
. The explicit formula of the bound for each estimand is given in Section B.1. I find that bounds for the two estimands diverge suggesting there are many observations that have Y i1 (1) = Y i1 (0) in the population. For example, among Democrats, the bound for τ is between around 0.8 and 1.0 which might suggest that proportion of Democrats who supports gun control when treated is extremely high. However, the bound for η is between around 0.2 and 0 which might suggest that proportion of Democrats who have strictly prefer a more strict gun control is very small. This is a problem discussed in Lu, Ding and Dasgupta (2018) where τ and η cannot be informative when there are many units who does not change attitudes by the treatment. Therefore, it appears that we need to turn to other estimators that avoid this problem (e.g., Chiba, 2017;Lu, 2018).

D.3 Different treatment cutoff
Although Newman and Hartman (2019) define the exposure by the 100-mile cutoff, Barney and Schaffner (2019) consider different threshold to assess the robustness of the results. Following their analysis, I consider an alternative threshold of 25 miles. Figure D.4 shows the result. Note that the "prior exposure" is also defined by the 25-mile cutoff.

D.4 Two year subset of three-wave panel
In this section, I present two additional results based on the two-wave panel from CCES. Figure D.5 shows estimated effects based on the sub-group analysis taking interactions between the prior exposure variable and partisan identification. In the figure, circles (triangles) show estimates of ∆ 2 (∆ 1 ) and thin (thick) line indicate 90% (95%) confidence intervals. Confidence intervals are computed based on block bootstraps (blocked at the zip code level). I find that among no-prior-exposure group, the effect is concentrated among Democrats (∆ 2 for Democrats is estimated positive and statistically different from zero at the 10% level, while ∆ 1 is not statistically significant). On the other hand, effects for Independents and Republicans are both indistinguishable from zero at the 10% level (neither ∆ 2 nor ∆ 1 ). I also find that effects are almost zero in the prior-exposure groups regardless of partisanship.

E Details of the Application
A list of method used in the original studies Table 1 summarizes the methods used in the original papers. Coding of mass shootings Newman and Hartman (2019) uses the following criteria to determine if an incident constitutes a mass public shooting: "(1) firearms as the primary weapon used, (2) attacks on non-family members of the general public and (3) attacks in which at least three or more individuals were injured or killed." (Newman and Hartman, 2019, p.8). See the original studies for the detail of why these criteria are selected. Note that the definition of the "treatment" is slightly different between Newman and Hartman (2019) and Barney and Schaffner (2019). I follow the definition used by Barney and Schaffner (2019); Please see Barney and Schaffner (2019) for the discussion on this point.

F Simulation Studies
In this section, I present two Monte Carlo studies to investigate finite sample performances of the proposed method. The first simulation assesses performance of the proposed estimator for the causal effect where I compare the proposed estimator against the standard differencein-differences with dichotomized outcome and the ordered probit regression. The result shows that the proposed estimator is unbiased to the causal effects and the confidence interval has nominal coverage, while the other two method are biased and thus the confidence intervals fail to maintain the coverage. The second simulation studies the finite sample performance of the proposed procedure for the diagnostic in Section 3.4. I demonstrate that the type I error is controlled under the range of equivalence threshold that is compatible with the null and that the power converges to one when the equivalence threshold is chosen reasonably.

F.1 Estimating causal effects
In this first simulation study, I investigate the finite sample performance of the proposed estimator under the correct model specification. The potential outcome is generated by first drawing the latent utilities from the normal distribution. For the potential outcome under the control, the following set of parameters are used to generate the data: θ 00 = (−0.5, 1.5) , θ 01 = (1, 1) and θ 10 = (−1.5, 2) . The parameters for the counterfactual outcome θ 11 is set according to the identification formula in Proposition 1. The parameters for generating the potential outcome under the treatment, Y i1 (1), are set as µ = 1.5 and σ = 1.5.
After generating the latent utilizes, they are transformed into categorical outcome with J categories based on the set of cutoffs. In this simulation, I consider J ∈ {3, 5, 7} and also I vary the number of units n ∈ {1000, 2500, 5000}.
Since there are J −1 possible treatment effects to consider, that is, {∆ j } J−1 j=1 , estimators are evaluated on averaging the loss over J − 1 treatment effect estimates. Specifically, I consider the following metrics: j is the estimate of ∆ j under sth Monte Carlo iteration and C (s) j,1−α/2 is the 100 × (1 − α/2)% confidence interval for ∆ j . Figure F.1 shows the result. Left panel shows the absolute bias of the estimate. We see that the bias is larger when the sample is relatively small for J = 5 as the number of observations in each category tend to be smaller. However, in general, estimates are unbiased. Middle panel shows the RMSE. It shows that RMSE decreases as the sample size increases and the variance is smaller when the number of categories are smaller. Finally, the right panel shows the coverage of 90% confidence intervals. We can see that for both cases, confidence intervals have nominal coverage regardless of sample size.
In order to assess how the test performs depending on a choice of equivalence thresholds, I vary δ. The value of δ is chosen such that in some range of δ, the null of t max ≥ δ is true and in other range of δ the null is false (i.e., t max < δ). For the range of δ that satisfies t max ≥ δ, I set δ ∈ {t max − 0.05, t max − 0.01, t max }. We would expect that the test is more likely to reject the alternative when δ = t max − 0.05. For the range of δ that does not satisfy t max ≥ δ, I set δ ∈ {t max + 0.05, t max + 0.01, t max + 0.10}. Among them, we expect that the test can reject the null most likely when δ = t max + 0.10. Figure F.2 shows the results for this simulation. The upper panels show type I errors when the choice of δ is consistent with the data (i.e., t max ≥ δ). Recall that the data is simulated such that the equivalence does not hold. Thus, we would expect that the null, H 0 : t max ≥ δ, is not rejected and the probability of falsely rejecting the null (type I error) should be less than α. In fact, we can see that the proposed testing procedure controls the type I error. In addition, the smaller value of δ (i.e., a smaller rejection region for H 0 ) leads to lower type I error. The lower panels show results for type II errors when the choice of δ is not consistent with the data (i.e., H 0 is false). We can see that the test struggles to reject the null when δ is set to close to t max . When δ is set to a value far away from t max , type II error converges to zero as sample size increases. under thresholds that are compatible with H 0 (i.e., H 0 is true: t max ≥ δ holds). The lower panel shows the power curve in a range of δ that is not compatible with H 0 (i.e., H 1 is true: t max < δ).