{'fraction_non_alphanumeric': 0.03386318329661106, 'fraction_numerical': 0.016393872626167243, 'mean_word_length': 5.053509050492219, 'pattern_counts': {'":': 1, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Quotatives Indicate Decline in Objectivity in U.S. Political News

According to journalistic standards, direct quotes should be attributed to sources with objective quotatives such as"said"and"told", as nonobjective quotatives, like"argued"and"insisted"would influence the readers' perception of the quote and the quoted person. In this paper, we analyze the adherence to this journalistic norm to study trends in objectivity in political news across U.S. outlets of different ideological leanings. We ask: 1) How has the usage of nonobjective quotatives evolved? and 2) How do news outlets use nonobjective quotatives when covering politicians of different parties? To answer these questions, we developed a dependency-parsing-based method to extract quotatives and applied it to Quotebank, a web-scale corpus of attributed quotes, obtaining nearly 7 million quotes, each enriched with the quoted speaker's political party and the ideological leaning of the outlet that published the quote. We find that while partisan outlets are the ones that most often use nonobjective quotatives, between 2013 and 2020, the outlets that increased their usage of nonobjective quotatives the most were"moderate"centrist news outlets (around 0.6 percentage points, or 20% in relative percentage over 7 years). Further, we find that outlets use nonobjective quotatives more often when quoting politicians of the opposing ideology (e.g., left-leaning outlets quoting Republicans), and that this"quotative bias"is rising at a swift pace, increasing up to 0.5 percentage points, or 25% in relative percentage, per year. These findings suggest an overall decline in journalistic objectivity in U.S. political news.

"Absolute" objectivity has been criticized as unattainable, as structural biases would creep into news production (Beharrell et al. 2009), or even harmful, as the excessive balance of viewpoints could create an illusion of credibility for dubious or unsupported positions (Dixon and Clarke 2013). However, amidst the fragmented media ecosystem that emerged from the digitization of news outlets and the algorithmic serving of content (Thurman, Lewis, and Kunert 2019), journalism scholars have argued that objectivity has become ever more important to consumers of journalism (Boudana 2011;McNair 2017). This opinion is also held by the public around the world, who, as of 2018, overwhelmingly agree that news media should be unbiased in its coverage of politics (Mitchell et al. 2018).
One of the concrete ways in which journalists have sought to report the news objectively is through the usage of direct quotes (Brooks et al. 2007;Stenvall 2008). Since journalists almost never directly observe the events they report, using quotes lends them more reliability and factuality than their own words (Van Dijk 1988). Further, direct quotes would let people "speak for themselves", following one of the golden rules of journalism (Ingram and Henshall 2012). However, in a direct quote, journalistic objectivity can still be compromised by the use of certain quotatives that relay the emotions of reporters to readers (Mencher and Shilton 1997) or the attempt to describe the speaker's state of mind (Gidengil and Everitt 2003). For example, in the direct quote objectivity in political news coverage can help inform new practices and interventions that seek to improve the political news media ecosystem. Quotatives, in this context, are a powerful instrument to measure bias. Studying how journalists deviate from the standard usage of quotatives -e.g., "say" and "tell" (The Associated Press 2020) -allows researchers to quantitatively assess adherence to journalistic objectivity (Lee 2017) and reveal biases in journalistic coverage of politics (Gidengil and Everitt 2003). Present Work. This paper analyzes quotatives to study objectivity and media bias in political journalism. We ask: • RQ1 How has the usage of nonobjective quotatives evolved in U.S. political journalism?
• RQ2 How do news outlets use nonobjective quotatives when covering politicians of different parties?
To answer these research questions, we developed a methodology to extract quotatives from a large-scale corpus. We then performed a comprehensive study on how (and which) quotatives are used in direct quotes from U.S. politicians between 2013 and 2020, leveraging a large dataset of quotes from English-language media linked with relevant speaker metadata (Vaucher et al. 2021) and enriched with the political leanings of different U.S. outlets. By counting the usage of nonobjective quotatives like "shout" or "assert", we analyze how U.S. news outlets of different political inclinations adhere to basic journalistic objectivity principles and how this adherence evolves. Further, analyzing how outlets of different political inclinations use quotatives to talk about politicians of different parties, we study the evolution of quotative bias in news outlets.
Summary of Findings. We find that the usage of nonobjective quotatives varies across different outlet categories. Overall, the more ideologically extreme an outlet is, the more nonobjective quotatives it uses. However, we also find centrist outlets are experiencing a significant increase in the usage of nonobjective quotatives over the last years (about 0.5 percentage points, or 15% in relative percentage), suggesting that they may be "catching up" to the more biased outlets, who are not experiencing such significant increases (RQ1). We also find evidence of "quotative bias", i.e., outlets tend to use nonobjective quotatives, especially when referring to politicians of opposing ideology. For instance, left and right-leaning outlets use nonobjective quotatives up to 2% more often when referring to republicans and democratic politicians, respectively (RQ2). Last, we find that this quotative bias is increasing at a swift pace, increasing as much as 0.5 percentage points per year in absolute percentage for left-leaning outlets, suggesting a rapid increase in polarization (RQ1 and RQ2).
Implications. Our findings indicate a decline in journalistic objectivity in U.S. political news, particularly from centrist outlets. This suggests that centrist outlets may play a role in the increasingly less respectful and fact-based debate around politics (Doherty et al. 2019). Further, we also find evidence of an increasing quotative bias, which could further erode trust in media (Brenan 2022).

Background and Related Work
When a quote occurs in the news, three elements are typically involved: the source, i.e., the speaker who uttered the quote (underlined in the examples); the quoted content itself (in italic); and the quotative that introduces the quote (also known as a cue, reporting verb, speech verb, or attribution verb; in bold). Quotes can be classified as either direct, indirect, mixed, or pure (Cappelen and Lepore 1997), where only the former three types are typically of concern to journalists. We give an example of a direct quote in the introduction and of mixed and indirect quotes below. In direct and mixed quotes, a pair of quotation marks are used, and we can infer that the speaker indeed uttered the quoted words, whereas indirect quotes may paraphrase the speaker's words. Therefore, journalists have the most freedom in word choice in indirect quotes, as they can, to some extent, rewrite what the speaker said. In contrast, in direct quotes, journalistic norms require them to report the quoted words verbatim (Harry 2014).
In our work, we focus on direct quotes, as Quotebank does not contain indirect quotes (Vaucher et al. 2021) and since automatically extracting the quotative in mixed quotes is technically challenging and, in some instances, impossible as there is no quotative, e.g., John will not help as he has "done more for this house than all of us combined".
Measuring Media Bias. Previous work has studied "media biases": how journalists' and editors' personal opinions, beliefs, and financial incentives shape what is considered newsworthy (McCombs and Shaw 1972) and how issues are covered (Iyengar 1994). Scholars argue that partisan media bias can harm democracy by distorting citizens' political knowledge and increasing polarization (Bernhardt, Krasa, and Polborn 2008;Boudana 2011;McNair 2017). Thus, measuring media bias is the first step to improving our information ecosystem (Watts, Rothschild, and Mobius 2021).
Early studies in media bias required extensive manual annotation. For instance, Kobre (1953) studied how the press in Florida covered the U.S. 1952 presidential campaign by coding the number of inches of text given to each party, the position of pictures, etc, across hundreds of newspapers. However, in recent times and with the digitization of news, various methods have been developed to automatically measure media bias (Hamborg, Donnay, and Gipp 2019). Some of these methods are audience based, measuring how segregated news consumers are across outlets, e.g., Zhou, Resnick, and Mei (2011) use votes on Digg, a social news aggregator, to classify political articles. Others are content based, quantifying media bias by analyzing published content directly. For instance, Gentzkow and Shapiro (2010) measured bias using the frequency at which outlets reproduce partisan phrases in congressional speeches.
According to Budak, Goel, and Rao (2016), content and audience-based approaches present their particular limitations. On the one hand, audience-based approaches do not scale beyond the mainstream outlets from which detailed readership information can be obtained. On the other hand, content-based approaches struggle to generalize well across different types of news and outlets, e.g., methods that try to match politicians' speeches to news only apply to a minority of news articles, limiting the scope of the results obtained with such methods.
Quotatives and Bias. Quotatives can impact how readers perceive a news story and the involved speakers (Geis 1987;Just, Crigler, and Buhr 1999). For instance, Cole and Shaw (1974) carried out an experiment in which they changed "objective" quotatives like "said" for stronger verbs like "argued" or "insisted" and asked participants to rate stories across a variety of criteria. Among other things, they found that in the modified versions, stories were perceived as more exciting and less objective, and speakers were perceived as more rash. Through quotatives, journalists can "paint reports on speech with any brush they like" (Geis 1987), which would both reveal the beliefs and preferences of the writer (Gidengil and Everitt 2003), but also subtly influence the reader (Cole and Shaw 1974). In this context, quotatives have been used to measure bias in politics, sometimes referred to as "attribution bias." This line of work dates from the 1960s when Merrill (1965) studied how Time magazine used quotatives (among other things) when referring to U.S. Presidents Kennedy, Truman, and Eisenhower. More recently, Gidengil and Everitt (2003) analyzed differences in quotative usage between male and female party leaders on Canadian television, finding that female leaders' speech was reported with more negative and aggressive quotatives. With a similar methodology, Lee (2017) studied differences in nonobjective quotatives between offline and online newspapers, finding the former to adhere better to journalistic standards.
Quote Attribution and Analysis. Previous work in natural language processing has extensively studied the problem of quote attribution (see Vaucher et al. (2021) for a comprehensive review), which is an important task in understanding dialogue structure and developing better conversational agents. For each quote, the goal is to extract the speaker of the quote, either at the mention level or entity level. This task is challenging as the speaker could be mentioned as an anaphora or implicitly. This task can be further combined with named entity linking to extract unique IDs of speakers (Čuljak et al. 2022). Most prior work, however, has not dealt with the problem of quotative extraction.
Nonetheless, several datasets annotated for attributional relationship exist (Pareti 2012(Pareti , 2016Newell, Margolin, and Ruths 2018) that could be considered in this context. These datasets contain labels for the content, source, and cue for each attributional relationship and can be viewed as an extension of The Penn Discourse TreeBank 2.0 (Prasad et al. 2008) that provides annotation of discourse relations and argument structures. While these datasets can potentially be used as resources for training a supervised model for quotative extraction, the attributional relationship they considered is much broader than quotation. Thus, we decide not to utilize them in this study.
Existing work has also analyzed quotes from different perspectives. Niculae et al. (2015) found a systematic pattern in the outlets' quoting behavior when covering the exact same event. Lazaridou, Krestel, and Naumann (2017) found that a machine learning classifier could reliably predict one of two news outlets based solely on the quotes they report, demonstrating media bias. Tan, Peng, and Smith (2018) showed a declining trend of bipartisan quote coverage with a bipartite graph of media outlets and the sentences they quoted. Külz et al. (2022) analyzed the quotes of U.S politicians between 2008 and 2020 and found a decrease in negativity during Obama's tenure and a sudden and lasting increase starting from Trump's presidential primary campaign in 2015.
Relationship with Prior Work. We develop an automated method to measure biases by analyzing quotatives (going beyond the manual annotation used in prior work). In light of the existing literature on automatically measuring media bias, our method can be understood as a content-based approach. Although it measures a specific type of bias (present in the quotative verbs), our approach does not share the limitation of other content-based methodologies for measuring media bias since it generalizes well across different news and outlets. Finally, our work expands the literature on automated quote analyses, as previous work did not study variations in the usage of quotative verbs over time.

Data and Data Processing
To study quotative usage across various news outlets, we use the Quotebank (Vaucher et al. 2021) dataset, a webscale corpus of quotes. Quotebank contains over 235 million unique quotes, extracted from 196 million English news articles from 377 thousand web domains between September 2008 and April 2020. We additionally obtain a list of current and former U.S. politicians with their party affiliation from Wikidata, in the same fashion as Külz et al. (2022), and filter QuoteBank to consider the period containing the best-quality speaker attributions (May 2013 to 2020), and retain only quotes from politicians on this list. In cases where quotes are attributed to more than one speaker in Quotebank, we heuristically assume the first attribution to be correct. We validate speaker attribution in our filtered dataset on a manually annotated sample of 100 ambiguous quote attributions and find this heuristic to be 86% accurate.
To ensure the validity of our findings, we preprocess Quotebank as depicted in Figure 1. We 1) use heuristics to retain only direct quotes; 2) extract quotatives and remove quotes without quotatives in the verb form; 3) filter quotes, keeping only those from U.S-based outlets with humanverified bias ratings; and 4) create dictionaries of common quotatives, removing quotes with rare quotative verbs for which quotative extraction performs poorly. We detail each of these steps in the following paragraphs.  Figure 1: Data processing pipeline. We outline the key steps in our data processing pipeline, as well as the percentage of retained data after filtering.
Step 1: Removing Titles and Mixed Quotes. To remove titles from the dataset that are erroneously recognized as quotes (e.g., movies, books, etc.), we apply a filter using the percentage of words in a quote whose first letter is capitalized (using a threshold of 50% capitalization). Afterward, to remove mixed quotes, we employ a sentence recognition filter that combines constituency parsing 2 and dependency parsing 3 . We retain only quotes that can be parsed as a full sentence at the root level by constituency parsing and contain a subject and a predicate (root) in dependency parsing. These heuristics greatly improve data quality (e.g., extracted quotatives in Step 2 are much more accurate) while retaining 76.2% of the dataset.
Step 2: Extracting Quotatives and Removing Non-verb Quotatives. In the next step, we adopt a three-stage approach to extract the quotative from each quote using dependency parsing. First, we run dependency parsing and acquire a distribution of quotatives from the root node of each parsed quote. Second, we add a condition such that when there is one verb being identified as the root and another verb in a parallel node 4 , we choose as the quotative the verb with the higher probability according to the distribution of verbs extracted in the first stage. Finally, we take the lemma of each extracted quotative and remove quotatives that are not in verb form. After this step, we retain around 75.4% of the original data.
2 Constituency parsing breaks down sentences into phrases and identifies their grammatical roles, e.g., in "I eat a big apple", "a big apple" is a noun phrase. See Jurafsky and Martin (2022) for details.
3 Dependency parsing extracts dependency relationships between words, with verbs typically being in the structural center, e.g., in "I eat a big apple", "big" is an adjectival modifier of "apple." See Kübler, McDonald, and Nivre (2009) Table 1: Final dataset statistics. Frequencies of the ten most frequent quotatives and speakers in the dataset after filtering. Among the listed quotatives, only "declare" is nonobjective.
Step 3: Removing Unsuitable Outlets. We obtain a list of media bias ratings from mediabiasfactcheck.com (hereinafter MB/FC) and classify outlets into 5 categories based on the bias rating: left, left-center, mainstream, right-center, and right. We remove quotes from outlets without a bias rating, from outlets that are not from the U.S. (also according to MB/FC data), and from outlets that have very few quotes (which may suggest data quality issues), only keeping outlets with more than 20 quotes over a period of 12 months. After this step, around 38.0% of the original data remains, all of which are from relevant U.S media outlets with humanverified bias ratings. Manual inspection of the removed data confirms that the removed outlets are predominantly nonnews websites, small local newspapers, radio stations, and non-U.S news outlets.
Step 4: Creating Dictionaries of Common Quotatives. Inspired by Lee (2017) as well as the recommendations laid out in Reuters (2008) and The Associated Press (2020), we define quotatives as objective if they refer to the direct speech action and do not involve any subjective judgment of the action (e.g., like "say" and "tell"); and as nonobjective if they refer to some additional action or conduct and with subjective judgments (such as "boasted", "rasped", "taunted", or "hailed"). To optimize for precision, we exclude common verbs with many non-quotative senses, such as "go." Using this definition, we manually annotate the most frequent 99.5% of quotatives overall and the 98.0% of the most frequent quotatives per month. This process yields a list of 35 objective verbs and 152 nonobjective verbs (c.f., Appendix A). We use this list to remove rare verbs (i.e., those not on the list), obtaining a final dataset with 6.7M quotes (35.1% of the original data) from 14,031 politicians in 989 outlets. Table 1 summarizes the most frequent quotatives and speakers in the final dataset. Consistent with prior literature (Gidengil and Everitt 2003), "say" and "tell" are the most common quotatives, used more than 80% of the time,  Table 2: Validation of the proposed quotative extraction method. We take 100 random quotes from each year that our dataset covers, manually annotate the correct quotative, and show the accuracy. Note that the incorrect cases include those in which a quotative cannot be identified. This is most often because no clear quotative is present (due to mixed quotes or because the article text is incomplete).
and Twitter is a common source for quotes (Bane 2019).

Validation Error Analysis
To validate our verb extraction approach, we divide our data based on the detected verb type (objective and nonobjective) and manually annotate 100 randomly sampled quotes per year per verb type; the results are shown in Table 2. Our method achieves high accuracy in objective and nonobjective verb extraction, and this accuracy is stable over time, with a combined accuracy of over 90% across all years. Before removing quotatives that are not contained our dictionary of common quotatives (see Step 4), we also analyze the instances where quotatives with rare verbs were retrieved by our model. We find that in many cases: 1) the actual quotative is not a verb (e.g., "according to" is a common quotative phrase); 2) the quote is a mixed quote without quotative; or 3) the method is applied to incomplete or noisy data and identifies an incorrect quotative.

RQ1: How Has the Usage of Nonobjective
Quotatives Evolved?
Across the study period, we find that the usage of nonobjective quotatives produces a sensible ordering of the media bias categories considered, with the more partisan outlets using the most nonobjective quotatives and the less partisan outlets using the least. We depict this order in Fig. 2  most. Pairwise differences between averages are statistically significant.
To study how the usage of nonobjective quotatives evolved, we use a fixed effects linear probability model. For each quote q in our dataset, let o[q] be the outlet in which q was reported, b[q] be the bias category of the outlet, and t[q] be the time when it was in reported in months relative to the starting period of our dataset (May 2013). We then define the model as where the dependent variable y q equals 1 if the verb used in the quote q is nonobjective and 0 otherwise, α o[q] is an outlet-level intercept, γ b[q] is a category-level intercept, and β b[q] is a category-level trend in the usage of nonobjective quotatives -what we are interested in estimating. 5 Since we are modeling time-series data (one per outlet), autocorrelation may shrink the confidence intervals of model estimates [see Bertrand, Duflo, and Mullainathan (2004) for details].
To address this, we estimate the model using cluster robust standard errors, clustering on the outlet level (Cameron and Miller 2015).
We depict the estimated trends in the usage of nonobjective quotatives in Fig. 4 [i.e., the estimated β b[q] in Equation (1)]. Although centrist outlets used nonobjective quotatives less on average (c.f., Fig. 2), we find that their usage of nonobjective quotatives increases over time. We estimate that mainstream outlets increase their usage of nonobjective quotatives by 0.08% per year and that right center and left center outlets increase their usage by 0.10% and 0.06% per year, respectively. Over the entire study period, these changes translate to relative increases of 19.9% for mainstream outlets, 21.3% for right center and 13.6% for left center outlets. In contrast, left outlets experience a statistically insignificant decrease in their usage of nonobjective quotatives by 0.12% per year, and right outlets experience a smaller, not statistically significant increase in usage of nonobjective quotatives (of roughly 0.01%). We further illustrate the results obtained in the fixed effects model in Figure 3. We center each outlet time series around 0 and then report the month-level (demeaned) usage of nonobjective quotatives per outlet category, along with a regression line capturing the trend in each time series. Here, we again see that the usage of nonobjective quotatives increases for centrist outlets, decreases for outlets on the left, and only slightly increases for outlets on the right, although the two latter results were not statistically significant according to the model. Another way to understand the change in quotative usage is to consider the extremes. We compare how quotative usage changes between the first 12 months (May 2013 -April 2014) and the last 12 months (May 2019 -April 2020) of our dataset. In Table 3, we report the quotatives that experienced the largest changes in terms of absolute percentage points (on the left) and odds ratio (on the right). We find that the usage of the quotative "say", considered the gold standard of quotatives, fell by more than 10% percentage points. At the same time, we see an increase in other objective quotatives (e.g., tell), but this increase does not account for the entire ten percentage points. Lower in the list, we see that nonobjective quotatives like claim, remark, and declare are used more often. Finally, we highlight that quotatives reveal changes in where journalist source their quotes, with both "tweet" and "caption" (usually employed when the speaker uploads a picture or video on social media) experiencing large relative increases in usage.  Table 3: Changes in quotatives used. We report changes in quotative usage between the first and last 12 months of our dataset in terms of absolute change and odds ratio. Italic highlighting denotes nonobjective quotatives.

RQ2: How Do News Outlets Use Nonobjective Quotatives When Covering Politicians of Different Parties?
Next, we investigate whether the outlets are biased in their quotative usage when they cover politicians from ideologically similar vs. opposing political parties. Quotative Bias Across Outlet Categories. For each quote q, let p[q] be the party of the politician who uttered the quote. Keeping with the notation in Equation (1), we again use a fixed effects linear probability model where the dependent variable y q equals 1 if the quotative used in the quote q is nonobjective and 0 otherwise, is a category-level intercept, η p[q] is a party-level intercept, and σ b[q],p [q] captures the interaction between pairs of outlet bias category (b[q]) and speaker party (p[q]). We again cluster standard errors on the outlet level to address autocorrelation. Note that here, we are particularly interested in the contrasts between different combinations of outlet categories and speaker parties, e.g., the difference between how left outlets quote Democratic and Republican speakers (in the model σ left,democratic − σ left,republican ).  Figure 5: Difference in nonobjective quotative usage between Democratic and Republican speakers. All estimates are significant. Reported differences correspond to the contrasts σ democratic − σ republican in Equation (2).
For each media bias category, we show the estimated difference in usage of nonobjective quotatives for Democratic and Republican speakers in Figure 5. We find that, for every bias category of media outlet, there is a significant difference in quotative usage between Democratic and Republican speakers. Notably, this difference is nearly 2%, around a third of the overall nonobjective quotative usage, for both left and right media outlets, which use more nonobjective quotatives when referring to politicians from opposing political ideologies. For centrist outlets, we see a Democratic bias in the usage of quotatives, with Republicans being quoted with nonobjective quotatives around 1% more for mainstream and left center outlets and nearly 0.5% more for right center outlets. These results are in accordance with previous work that has found an overall liberal bias across mainstream outlets (Groseclose and Milyo 2005).
Matched Analysis. A possible explanation for what we observe in Figure 5 is that outlets of different political leanings cover different quotes (Tan, Peng, and Smith 2018), and that these quotes lend themselves more or less to being attributed to speakers through nonobjective quotatives. Even if this were true, one could still make a case for media bias, as journalistic textbooks and guidelines instruct the usage of objective quotatives regardless of the quote (Mencher and Shilton 1997;Brooks et al. 2007;Reuters 2008;The Associated Press 2020;Rich 2015). Nevertheless, we entertain this hypothesis by performing a matched analysis. Specifically, we identify quotes covered by both the left and right media outlets, merging left/left center and right/right center outlets for ease of comparison (we refer to these merged bias categories as left and right "combined"). Using only this subset of quotes (n=1.02M, 15.13% of all quotes in our data), we fit the fixed effects model defined in Equation (2).
For the two collapsed media bias categories and considering only matched quotes, we show the estimated difference in usage of nonobjective quotative for Democratic and Republican speakers in Figure 6. For right/right center outlets, we find a small non-significant positive difference (0.01%) that may be due to heterogeneous trends between right and right center outlets (which differ in the non-matched analysis, c.f., Figure 5). For left/left center outlets, we find a significant difference of around -0.75% in the usage of nonob-  Figure 6: Difference in nonobjective quotative usage between Democratic and Republican speakers for matched quotes. A solid circle denotes a significant effect (p<0.05) and a hollow circle denotes an insignificant effect. Reported differences correspond to the contrasts σ democratic − σ republican in Equation (2).  Figure 7: Difference in percentage yearly rate of change of nonobjective quotative usage between Democratic and Republican speakers for each media outlet category. A solid circle denotes a significant effect (p<0.05) and a hollow circle denotes an insignificant effect. Reported trends correspond to the estimated λ coefficients in Equation (3) jective quotatives, suggesting that quote selection alone cannot explain the quotative bias previously observed. Trends in Quotative Bias. Finally, we investigate if quotative bias has evolved during the study period, using a fixed effects linear probability model: where the dependent variable y q equals 1 if the quotative used in the quote q is nonobjective and 0 otherwise, q] , and η p[q] are outlet, category, and party-level intercepts, and λ b[q],p[q] is the trend in usage of nonobjective quotatives for each party/bias category combination. For each media bias category, we depict the difference in the trends of nonobjective quotative usage for Democrats and Republicans in Figure 7. We find that for left and centrist outlets, the gap between how nonobjective quotatives are used to quote Democrats and Republicans is increasing in the study period. These increases are not only statistically significant, but also substantial in comparison with the existing level of quotative bias observed in our data. For example, the estimated contrast of the trend is of around 0.33% for left center outlets, and the existing quotative bias is 1.08%. Thus, the annual relative increase of quotative bias is above 30%. Left outlets exhibit the most increase in quotative bias in absolute terms, at 0.5% per year. For right outlets, we find this difference in trends leans towards Republicans, but the effect is not statistically significant.
In this work, we proposed a method to automatically identify quotatives for direct quotes using dependency parsing. We then extracted quotatives from a large dataset of speakerattributed quotes, resulting in over 6.7 million quotes over 8 years, from 2013 to 2020. By counting the usage of objective and nonobjective quotatives, we analyzed the static and dynamic trends of quotative usage.
We find that the more partisan outlets use more nonobjective quotatives (Figure 2). However, during the study period, centrist outlets (classified as mainstream, left center, and right center by MB/FC) experience a significant increase in the usage of nonobjective quotatives, suggesting that they may be "catching up" to the more biased outlets (Figure 3). We further observe that outlets tend to use more nonobjective quotatives when covering politicians of the opposing ideology, thereby exhibiting "quotative bias" (Figure 5). Even when we control for quotes, matching outlets on the quote level, we find that this bias still exists for left and leftcenter outlets ( Figure 6). Last, we find a rapid increase in quotative bias for most outlet categories over time, which may suggest that U.S. political news is becoming increasingly polarized (Figure 7).
These findings suggest that two simultaneous processes are at play: outlets are adopting more nonobjective quotatives overall and outlets that the usage of nonobjective quotatives is increasingly "mediated" by the party affiliation of quoted politicians. Both processes indicate a measurable decrease in journalistic objectivity. While detecting bias often requires some level of human judgment to determine neutrality, and while it is debatable how a neutral or balanced view can be presented in any specific context, quotative usage can be regarded as an easily quantifiable form of bias due to its prominence within journalism. There are clear and established rules for the usage of quotes on which journalists have historically agreed, as is evident from textbooks (Brooks et al. 2007;Mencher and Shilton 1997;Rich 2015) and editorial guidelines (Reuters 2008; The Associated Press 2020). Although objective journalism is a 20th-century invention and could be considered an anomaly throughout journalism history, it is commonly regarded as central to the democratic process. In that context, our results indicate a decrease in the level of objective quotative usage in U.S political news coverage, which can be seen as a devolution of journalism as a profession.
Viewed from the consumers' perspective, quotative bias is a subtle form of media bias that affects the readers' perception of the story, and involved entities (Cole and Shaw 1974;Geis 1987;Just, Crigler, and Buhr 1999;Gidengil and Everitt 2003), but is nevertheless noticeable. Suppose otherwise objective mainstream media outlets exude quotative bias at an increasingly faster pace. In that case, it may lead already suspicious readers to further increase their perception of media bias in general (Gunther 1992), contributing to the erosion of trust in media.
One interesting question is to which extent Donald Trump, the most quoted speaker in our dataset, influences the findings observed in this paper. To investigate this, we rerun all fixed effects models in the paper in a filtered dataset with no Trump quotes (Appendix A.2). We find that, without Trump, there is no significant increase in the overall usage of nonobjective quotatives. However, we still observe quotative bias in all outlet categories except left center ones.
Limitations. We highlight three limitations of our work. First, while QuoteBank covers large number of outlets over time, the number of quotes from each outlet does not necessarily reflect the actual amount of content per outlet, nor does it represent the relative popularity of each outlet. Therefore, our findings should not be interpreted as the perception that an average news consumer in the U.S. may have. Second, since we focus on quotative verbs and do not consider adverbs, we may not capture the complete picture of quotative usage. For example, the hypothetical quotative "say aggressively" should be categorized as nonobjective but we still categorize it as objective since we do not consider adverb usage. Empirically, however, this combination of an objective quotative verb and a nonobjective adverb is uncommon (as it can be considered even more unprofessional than the nonobjective verbs) and should not affect our results. The occurrence of non-verb quotatives is rare as well. Also, despite our best effort, there are a small number of quotatives that are misidentified or simply not included in our dictionary. While this should not affect our overall analysis, analyzing these rare quotative might yield additional insights into how media outlets use quotatives. Third, while we carefully removed most indirect and mixed quotes from our dataset, some are expected to remain in the data. Thus, we cannot completely eliminate the effect of indirect quotes on the obtained results.
Future Work. Future work could use and extend our methodology to investigate trends in nonobjective quotative usage after the Trump presidency, or in other countries and languages. Further, an even more comprehensive investigation of the landscape of quotative usage in political journalism could be obtained by extending our methodology to include indirect and mixed quotes, and/or considering nonverb phrases and adverbs as quotatives.
Ethical Considerations. In this work, we use a publicly available data to analyze quotes from U.S politicians to study media bias. We match politicians' names between Quotebank and Wikidata. Our study utilizes one identity characteristic -political party of the involved speakers -at an aggregated level in order to investigate the effect of ideology in quotative usage. We do not perform any individuallevel inferences. Additionally, we manually validate both speaker disambiguation and quotative extraction methods to minimize the risk of identifying the wrong individuals and ensure the veracity of our findings. Given that politicians are public figures and the importance of research to better understand languages used in political journalism and its implications, we believe that our work is in line with reasonable expectations of privacy (Doherty 2007). We do not foresee potential negative societal impacts coming from this research. On the contrary, we believe that a better understanding of our political media ecosystem is essential to improve it. We confirm that we have read and abide by the AAAI code of conduct.