{'fraction_non_alphanumeric': 0.025713638423198914, 'fraction_numerical': 0.016594925237879473, 'mean_word_length': 4.909303882195449, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
After Sandy Hook Elementary: A Year in the Gun Control Debate on Twitter

The mass shooting at Sandy Hook elementary school on December 14, 2012 catalyzed a year of active debate and legislation on gun control in the United States. Social media hosted an active public discussion where people expressed their support and opposition to a variety of issues surrounding gun legislation. In this paper, we show how a content-based analysis of Twitter data can provide insights and understanding into this debate. We estimate the relative support and opposition to gun control measures, along with a topic analysis of each camp by analyzing over 70 million gun-related tweets from 2013. We focus on spikes in conversation surrounding major events related to guns throughout the year. Our general approach can be applied to other important public health and political issues to analyze the prevalence and nature of public opinion.


INTRODUCTION
Gun control in the United States is a major public policy issue that has polarized US society [26]. Although public opinion has been strongly in favor of stricter gun control policies for over two decades [29], federal gun control legislation has been a hotly contested issue meeting little legislative success, where even local restrictions have been met with opposition [31]. Insofar as public opinion affects the bills debated and passed into law, accurately gauging public opinion and salience on the various issues associated with gun control is important to inform the legislative process [6,18,21].
Public opinion is typically estimated through written or telephone surveys where subjects are asked to share their level of approval of different policies up for debate [2,15]. Assuming the population is uniformly sampled and that subjects are able and willing to divulge their true beliefs, these are reliable proxies for public opinion. Gun control polls are often conducted over the phone and ask respondents about their gun ownership, as well as opinions on different forms of gun control legislation (e.g., "Saturday night special" bans, assault weapon bans, national firearm registration, universal Bloomberg Data for Good Exchange Conference. 25-Sep-2016, New York City, NY, USA. background checks) [29].
However, traditional surveys have a number of drawbacks, including limitations on the response types and cost restrictions on producing timely results. These limitations are well known in the public health realm where surveys, a critical data source for a variety of public health topics, are facing increasing feasibility challenges. As a result, researchers have turned to new data sources, such as search queries 1 [10] and social media [7]. Social media has been used to estimate public opinion on a range of topics, including political sentiment [4,20,25,28] and a range of public health topics [3], including gun control [1]. Some work has looked at gun control tweets, but has focused on argument framing and not measuring public opinion [27].
Issues of gun control came to the forefront of national discussion with the mass shooting at Sandy Hook Elementary School in Newtown, Connecticut on December 14, 2012. This tragedy followed six months after another mass shooting in an Aurora, Colorado movie theater, and prompted a concerted effort to pass stronger gun restrictions at the federal level. In April, 2013, a bill to expand background checks was defeated in the senate, ending federal legislative efforts. Failure to pass national gun control legislation led many states, including Colorado and Connecticut, to pass their own gun control bills.
Public opinion played a major role throughout this time period, where discussions of gun control on social media rose in prevalence and prominence. The richness of social media data, where we have both overall prevalence, content and location data, presents new opportunities for analyzing and understanding the nature of public opinion surrounding guns.
We present an analysis of gun-related Twitter data from all of 2013, over 70 million tweets in total. We focus on two main questions: 1) Do Twitter conversations in support of or opposition to gun control reflect public opinion as measured by traditional surveys? 2) What events generate online activity from gun control supporters and opponents Table 1: Keywords used to collect tweets are listed as General keywords, and hashtags suggesting a Control or Rights gun control stance. and how do the arguments and issues discussed change in response to these events? While there has been significant work addressing our first question in regards to other topics of public opinion [20,3], the second question gives us a new framing in terms of social media studies; we are concerned with what social media users are saying about gun control, in addition to how many people are saying it.

METHODS
Our data set contains 70,514,588 publicly-available tweets collected using the Twitter streaming API based on keywords and phrases associated with guns or gun control in the United States: gun, guns, second amendment, 2nd amendment, firearm, firearms. Our collection covers just over one year, starting on December 16, 2012 (two days after the Sandy Hook shooting) and ending on December 31, 2013.
We identified hashtags indicative of support for (Control ) or opposition to (Rights) gun-control as a rough estimate of sentiment towards gun control. These hashtags were strongly associated with either the Control or Rights gun control positions. We obtained this list by examining the most popular hashtags in a subset of our data and selecting those that strongly indicated either one of these positions. Table 1 shows these hashtags: 11 for Control and 11 for Rights. A tweet was labelled as Control gun control if it contained more Control hashtags than Rights, and viceversa for Rights tweets. A total of 304,142 tweets were labelled as Control and 125,936 as Rights using this method. Although only about 0.6% of tweets were coded with gun control stance, this labelling method resulted in a high precision coding of tweets by gun control stance. We leave to future work statistical methods that identify gun control sentiment of a larger percentage of our data [14,30].
Our content analysis of the sentiment coded tweets relies on latent Dirichlet allocation (LDA) [5], a data-driven probabilistic topic model that can identify the major thematic elements in a text corpus. Topic models infer the parameters of a probability distribution with Bayesian priors, producing for each topic a distribution over the words in the corpus. Reviewing the most probable words for each topic is a common technique for establishing a semantically grounded label for the topic. Additionally, the model assigns a distribution over topics to each document (tweet), which enables  the tracking of topic proportions in a corpus over time [11]. Topic models have become popular tools for analyzing text data in social science [12], the humanities [19,17] and health [22,23], with numerous examples of applications to Twitter data [13,24,32].
We sub-sampled 6 million tweets (8.5% of the total collection) to train an LDA model, and then used the learned parameters to infer document specific topic distributions for each tweet. Tweets were tokenized by non-alphanumeric characters into unigrams and filtered using a stopword list specific to Twitter. We retained the 40,000 most frequent word types for learning. We used the LDA implementation in Mallet [16] and tuned model parameters on a held out set of 1 million tweets to maximize model log-likelihood. We swept the number of topics from 25 to 500, and the document-topic Dirichlet prior hyper-parameter α from 0.25 to 10 (with an asymmetric prior.) We used Mallet's parallel Gibbs sampler with a burn-in of 100 iterations, 500 total iterations, with hyper-parameter optimization every 10 iterations. Our tuned model used an initial α = 1 and 250 topics. The final model was then used to infer topics for the entire corpus using 200 sampling iterations.
We obtained a location for each tweet using Carmen [8], a high-precision geocoder for Twitter based on a user's profile. Wherever possible, we obtained the US state associated with a tweet. We chose to rely on an automatic geocoder since the proportion of tweets with location information provided by Twitter was small (around 1-2%.) Using the sentiment coded tweets and their inferred topic distributions, we measured the following trends. 1) The overall number of gun-related tweets for each day and week during 2013.
2) The number of Control and Rights messages for each day and week. Since the overall Twitter volume remained relatively stable in 2013, our counts are not normalized.
3) The most likely topics associated with Control or Rights tweets over the entire corpus, as well as for each week. This gives us a fine-grained look at which topics were discussed by each gun control camp for each week. We compute these trends for both the entire United States and for each US state.

Comparison with Polling Data
We begin by measuring the ability of Twitter to track gun related opinions as compared to results from traditional sur-  While our sentiment coding of tweets was for a coarse Control /Rights position on gun control, the polls do not directly ask this question. Therefore, as a proxy we selected the following question which appeared in all polls: "Would you support or oppose requiring background checks for all gun sales, including gun shows and the Internet?".
We used the proportion of "yes" answers from each poll as the value for each US state. For states that had two polls, we used each poll as a separate data point in our correlation. For Twitter, we measured the proportion of Control tweets over the number of both Control and Rights tweets for each US state over our entire collection. Due to data sparsity, we did not limit the tweets to consider only those from the time period the poll was taken.
We obtained a Pearson correlation coefficient of 0.51 between our two variables: -proportion "yes" in state polls and proportion of Control tweets. Figure 1 displays the leastsquares fit between these two variables, with an R 2 value of 0.22. This is a reasonably strong relationship between the variables, demonstrating that relative proportion of buzz in gun conversations on Twitter are reflective of opinions of the actual population.
This reasonably strong relationship was obtained even with several important limitations on our method. First, public opinion varied over time [9], yet these state polls capture just a single point in time, and different time periods at that. The time period of our Twitter data was mismatched to these polls, in that we used tweets from the entire corpus instead of restricting them to the time when the poll was conducted. Doing so would have yielded too few tweets, though future work that expanded our sentiment classifier method could address this problem. Second, we were only able to obtain polls and sufficient tweets for some US states, which reduces our ability to validate this method over the entire United States. Even though this was a major issue in US politics for a sustained period of time, polls were not conducted for every state. Third, gun control opinions can be complex, yet we are measuring only a coarse level of sentiment. The complex opinions expressed on Twitter may not map directly to our selected question. Fourth, we counted tweets, not the number of accounts tweeting. A single prolific account could bias our estimates. Finally, additional errors could be introduced by the accuracy of the geocoder, Twitter's representativeness of the US population, and the biases and sampling errors inherent in surveys. Despite these limitations, the obtained correlation is a strong indicator of the value of Twitter data for opinion analysis.

Opinions Surrounding Events
We next contextualize opinions as expressed on Twitter within the context of major gun control related events during 2013. We identified significant spikes in activity using the weekly aggregated statistics of Control and Rights tweets. For each spike, we used a historical news collection to identify major gun related events corresponding to the spike. Figure 2 displays Twitter traffic per week by Control and Rights with large spikes annotated with co-occurring events. Events of note are: • President Obama promises stronger national gun control legislation -Control tweets spike (December 19, 2012) 3 .
• The first gun control senate hearing featuring appearances from Gabrielle Giffords and Wayne LaPierre -  Control tweets spike (January 30, 2013) 4 .
• Connecticut passes strict gun control legislation in response to the Sandy Hook shooting -Control tweets spike (April 4, 2013) 5 .
• A compromise is reached over the gun control bill, significantly weakening the bill (April 10, 2013) 6 . Subsequently the push for stricter gun control was defeated in the senate (April 17, 2013) 7 . Rights activity spikes in both cases.
• Colorado gun control legislation banning high-capacity magazines goes into effect (July 1, 2013) 8 . Tweets increase for both Control and Rights advocates, although to a lesser degree than when national gun control legislation was being debated.
In our corpus, on average, Control tweets are much more common. Rights tweets eclipsed those of Control when gun control legislation failed to pass in April.

Major Topics of Discussion
Beyond detecting the overall sentiment surrounding each event, we characterized the content of each side by examining the topics discovered by the topic model. We selected the ten most likely topics for both the Control and Rights tweets over the entire time period: 304,142 Control tweets   Tables 3 and 4 show these topics, their likelihood, the most likely words, and our assigned label.
These topics and their relative order summarize the main thrusts of the conversation for both the Control and Rights group. For example, topic 237 centers around the group "Moms Demand Action for Gun Sense in America" and topic 246 around universal background checks -both topics prevalent in Control tweets. Topic 6 discusses armed robbery (presumably as an argument against new gun restrictions that would prevent citizens from protecting themselves) and topic 5 contains language indicative of political conservatives and second amendment rights advocates, in general.
We next contextualized these topics within the events described above. We computed the distribution over the 10 topics for the Control tweets and the 10 topics for the Rights tweets in the week around the event. By comparing how the usage of these topics change for each event, we can compute the dominant topics of conversation around each event. Figure 3 shows the relative proportion of the top 10 topics, overall for the set of Control and Rights tweets independently, as well as their proportion during each of the events. Topics are ordered by the their relative proportion over all Control or Rights tweets.
When President Obama initially promised federal gun control legislation, gun control advocates tweeted much more frequently about it, but this was not as prevalent during most other events, or overall. Universal background checks and models of more restrictive gun control policy are also mentioned much more frequently during the first senate hearing on gun control. Bloomberg gun control ads (2) Entertainment (225) Assault weapons ban (75) Gun bill filibuster (227) Second amendment (5) Gun defense anecdotes (6) Conservative hashtags/misc. (170) Local gun laws (121) Conservative hashtags/gun registry (69) Gun Rights When federal gun control legislation was first promised, gun rights tweets centered mostly around self-defense applications and state laws permitting carrying guns. During the first senate hearing on gun control, discussion also focused more on restrictions on assault weapons. As time progressed, former secret service agent and Republican political candidate, Dan Bongino became more vocal about gun rights. This is reflected in a greater proportion of tweets mentioning him.

DISCUSSION
By analyzing a year's worth of tweets on guns in the United States, we find variation in each side's reaction to gun related events, as well as variation in the arguments cited by each group during events of interest. Control advocates are very vocal early on in the debate when national legislation is still a possibility, but die down later on. From Figure 3, it is clear that a large proportion of this chatter was about national gun control legislation (Topic 7). Rights advocates became more vocal once the national legislation for universal background checks failed in congress, and much of their subsequent discourse focused on an assault weapons ban (Topic 75), the senate filibuster (Topic 227), and political candidate and gun rights advocate Dan Bongino (Topic 99).
We believe that this style of social media analysis is a complement to traditional polling techniques, which typically gauge opinion on a small set of issues. By fitting a topic model to the entire collection of gun-related tweets in 2013, we are able to identify salient issues and arguments for both camps, which researchers may not have identified as relevant, a priori. Most importantly, other than the keywords we searched for to collect this dataset and hashtags we used to label Control and Rights gun control tweets, there was no tailoring of our analysis to the gun control domain. This method of social media analysis can be applied to a wide range of salient public policy issues.