{'fraction_non_alphanumeric': 0.03318768951603276, 'fraction_numerical': 0.01446887758756536, 'mean_word_length': 5.170300530347673, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Test of the “News Diversity” Standard

The present study adds to the framing literature by providing a test of Mauro Porto’s “News Diversity” standard for professional journalism: providing multiple, equally persuasive frames in every news story. What are the effects of multiple versus single media frames on a foreign policy issue, and how do they interact with people’s foreign policy values? Previous studies have looked at how single frames influence opinions, how frames interact with values, and a few studies have investigated the effects of two competing frames at varying levels of persuasiveness. None, however, have used four, equally strong frames on a foreign policy issue and measured their effects among those with differing foreign policy values. We found that exposure to only single frames tended to move participants away from their stated values and in the direction of the frame, while exposure to multiple, competing frames kept participants’ opinions closer to their stated values—without merely reinforcing previously held opinions.

Foreign policy matters are no exception, as opinions on foreign policy that lack support among political elites do not tend to receive much media attention, keeping the public ignorant and policy options off the table (Powlick and Katz 1998). Concerns like these bring to mind Robert Dahl's (1959: 29) pessimistic conjecture that if "political preferences are simply plugged into the system by leaders (business or other) in order to extract what they want from the system, then the model of plebiscitary democracy is substantially equivalent to the model of totalitarian rule." Noting that normative democratic theory is only beginning to grapple with the amply demonstrated power of the media to influence citizens' opinions, Chong and Druckman (2013: 15) argue that the "endogeneity of public preferences calls into question the basic responsiveness model of democracy and forces us to reconsider the conditions in which the public can exercise autonomy and provide meaningful input in the democratic process." Motivated by such concerns, Mauro Porto (2007a) has argued for a "News Diversity" standard for journalism. Based on the interpretive citizen model, Porto defines "diversity" in the news media as a news environment containing a plurality of competing "interpretive frames," with the news media-as opposed to political parties and candidates-acting as the primary force in "shaping the menu of choices" (Porto 2007a: 311-12). "Interpretive frames" is the key element to the News Diversity standard. These frames have "specific 'sponsors,' the various social actors that promote specific interpretations of political reality, including politicians, organizations, and social movements" (Porto 2007a: 312). According to the News Diversity standard, major media outlets should present a broad variety of frames for all political issues, looking beyond the frames preferred by political elites to those presented by civil society groups and social movements. Furthermore, this diversity of frames should be present in every news story: even (or especially) when political elites are in agreement, the media should seek out dissident views and present them fairly alongside the dominant perspective(s). This would help ensure that debate in the modern, mediatized public sphere is not monopolized by powerful elites, and that underprivileged groups and marginalized views at least get to be heard and considered by the public.
However, the argument for a News Diversity standard hinges on whether exposure to multiple, equally strong frames actually results in the formation of less unduly influenced or polarized opinions. There would be no point in adopting such a journalistic standard if news consumers exhibit motivated reasoning by ignoring or discounting frames they dislike and by strengthening their opinions under the influence of the frames they do like. Studies of ideological polarization suggest the possibility that exposure to multiple frames may only increase levels of skepticism and spur news consumers to argue against frames they dislike (Taber and Lodge 2006), especially for those with higher levels of political knowledge (Taber et al. 2009). If this conjecture is correct, exposure to multiple frames may not result in a more informed public debate, but rather in greater polarization of opinions as motivated reasoning pushes all sides of a political debate to accept consonant and reject inconsonant frames. However, motivated reasoning faces significant constraints, including knowledge that contradicts a desired conclusion (Kunda 1990). That is, awareness of facts that would contradict a desired conclusion impedes the process of motivated reasoning to arrive at that conclusion.
Unfortunately, most framing studies have largely ignored how multiple, conflicting frames influence opinion: Only 3.2 percent of the 347 studies in Borah's (2011) review of the framing literature analyzed effects of competing frames. Even fewer have analyzed more than two, equally persuasive frames; we were unable to find any study providing participants with three or more equally persuasive but conflicting frames covering the same issue. The present study seeks to fill this gap by comparing the effects on opinion of four, equally persuasive frames versus single frames on a foreign policy issue.
Results from research on dual frames have shown that they tend to mute one-sided framing effects (Chong and Druckman 2007). While competing frames appear to cancel themselves out on average (Aklin and Urpelainen 2013;Wise and Brewer 2010), they provoke more consideration about the issue (Brewer and Gross 2005), and they can spur even the unmotivated to deliberate on an issue (Druckman 2011). Competing frames of an issue allow citizens to develop opinions on issues more in line with their values (Hansen 2007;Sniderman and Theriault 2004). A widely available, diverse array of frames can have direct effects on elections. For instance, during the 2002 Brazilian presidential election, legally mandated free television time afforded to the candidates provided new frames of the issues that conflicted with the dominant frames in the media-swaying the election in favor of a candidate disliked by the economic elite (Porto 2007b). New media may provide a greater diversity of frames (Im et al. 2011), suggesting that those who get their news primarily from the Internet may be less susceptible to one-sided framing effects simply by being exposed to a greater diversity of frames. Therefore, we expected that single frames would move participants' Ukraine policy opinions toward the frame and away from their foreign policy values (Hypothesis 1 [H1]), and consistent with Porto (2007a), we expected that exposure to multiple frames would keep Ukraine policy opinions closer to participants' foreign policy values, without polarization (Hypothesis 2 [H2]).

Foreign Policy, Values, and the Ukraine Conflict
Framing is quite effective concerning unfamiliar issues or events, or when it links familiar issues with existing beliefs, attitudes, and values (Tewksbury and Scheufele 2009). Values have been found to influence the effects of frames, with frames emphasizing considerations central to one's values being more persuasive (Nelson and Garst 2005), particularly among the more educated (Barker 2005). Frames can also be designed strategically to take advantage of competing sets of values (Nelson et al. 2011) or to activate certain values in the processing of a news story (Shah et al. 1996). Influential frames often tap into deep-seated cultural narratives, nudging us to interpret an event or issue in accordance with widely held beliefs, like "my country always seeks to do good around the world" (Lakoff 2011). Also, our preexisting knowledge about an issue-organized into schemas-moderates the effects of framing (Shen 2004). If we have well-developed, elaborate schemas on an issue, we are less likely to be influenced by a media frame that contradicts them (Shen and Edwards 2005). Frames may be most persuasive for those at medium levels of knowledge about an issue; those with little knowledge may be unable to incorporate the frame's message into their meager issuerelated schemas, and those with high levels of knowledge tend to be unlikely to modify their dense, highly structured schemas on the issue (Baden and Lecheler 2012;Zaller 1992). However, framing studies are contradictory on this point (Borah 2011), and the interaction between frames and knowledge likely varies across issue areas. We expected those with more strongly held foreign policy values and more knowledge about Ukraine to be less influenced by frames (Hypothesis 3 [H3]).
Values have been defined as ideas about what is desirable and how to achieve desirable outcomes; however, values are not all-powerful, and other factors in our "cultural tool-kits" influence our opinions and how we attempt to accomplish desirable goals (Hitlin and Pinkston 2013;Wuthnow 2008). Political values seem to emanate from personal values; for instance, personal values for security, conformity, and tradition correlate with political values for law and order, blind patriotism, and traditional morality (Schwartz et al. 2010;Schwartz et al. 2014). Political values concerning domestic policy are also strongly linked with political values concerning foreign policy (Holsti and Rosenau 1996;Rathbun 2007). In the United States, foreign policy values are relatively stable (Eichenberg 2007), remaining much the same even after significant changes like the end of the Cold War (Holsti and Rosenau 1993)-although in time some changes have emerged (Rosati and Creed 1997).
Scholars have used many different schemes to organize foreign policy values, including a two-orientation scale comprising hierarchy and community (Rathbun 2007), a three-orientation scale comprising militarism, anticommunism, and isolationism (Hurwitz and Peffley 1987), a four-orientation scale on the two dimensions of militant versus cooperative internationalism (Holsti and Rosenau 1990), and six- (Rosati and Creed 1997), seven- (Baum and Groeling 2009), and nine-orientation scales (Dolan 2008). For simplicity's sake, we avoided using the scales with higher numbers of orientations, but we also wanted to avoid over-simplification: Even on purely domestic issues, general Left-Right orientations differ on the dimension of social versus economic policy (Uhlmann et al. 2009). Therefore, we chose to use a two-dimensional scale producing four foreign policy orientations, combining aspects of the scales used by Hurwitz and Peffley (1987) and Holsti and Rosenau (1990).
Our scale involved a Left-Right dimension (supports vs. opposes helping outgroups/foreigners as well as the in-group/fellow citizens) and an Interventionism-Isolationism dimension (supports vs. opposes foreign military intervention), producing four orientations: Leftwing Isolationism (LISO), Leftwing Interventionism (LINT), Rightwing Interventionism (RINT), and Rightwing Isolationism (RISO). (For sake of comparison with Holsti and Rosenau's (1990) classifications, LISO corresponds with Accommodationists, LINT with Internationalists, RINT with Hardliners, and RISO with Isolationists.) Left Isolationists (LISO) would tend to believe in a more open, inclusive framework for international relations, to be more idealistic, and to want to help the world become a better place through diplomacy and economic aid. Left Interventionists (LINT) also believe in a more open, inclusive framework for international relations and are idealistic, but differ from LISO in that they are agreeable to the possible use of military force. Right Interventionists (RINT) tend to believe that the United States needs to be directly involved in world affairs and should use its economic, diplomatic, and military power to maintain world order and protect its national interests around the world. Right Isolationists (RISO) tend to believe that the United States should maintain its dominant position as the world's most powerful nation by protecting the homeland, avoiding foreign entanglements, and focusing on strengthening its economy. We have also provided a more detailed description of the four orientations in the appendices (see Online Appendix A).
Foreign policy value orientations could be divided into a nearly infinite number of classifications, and while our four-orientation classification is tractable and coherent, it sacrifices distinctions that exist within each orientation. (For instance, whether Right Interventionists prefer to build international coalitions or "go it alone," or whether Left Isolationists would support military intervention with U.N. Security Council approval.) Therefore, these four orientations should not be taken as measuring something essential or irremediable about the ways people think about foreign policy, but rather as one way in which to organize a highly diverse set of values and ideational influences-as such, we would not expect everyone to adhere strictly and exclusively to one of these four orientations, but more commonly to hybrid combinations of the four. Our classification was also chosen to provide a good fit with our review of press coverage of the conflict in Ukraine: The sources we examined in April 2015 could fairly comfortably fit within these four orientations (see Online Appendix A).

Participants
We collected an independent MTurk sample (n = 111) to rate the persuasiveness and familiarity of the articles we designed. We also collected a second MTurk sample for the experiment (n = 556, after eliminating participants who failed the attention check or spent less than fifteen seconds reading the article). Although not nationally representative, MTurk samples are more diverse than other Internet and college student samples (Buhrmester et al. 2011), although its subject pool tends to be younger and more liberal than the general U.S. population and includes a disproportionately large percentage of whites (Berinsky et al. 2012). As such, results cannot necessarily be generalized to the nation or beyond: 52 percent were female, 76 percent were white, and their ages ranged from seventeen to seventy-seven (M = 36.1 years, SD = 13.01 years). All participants were randomly assigned to control, LISO, LINT, RINT, RISO, and multiple-frame conditions in either Ukraine or "masked" format.

Materials
We designed newspaper articles that summarized the main points of each of the LISO, LINT, RINT, and RISO frames, a control article that presented bland facts about the conflict without an overarching frame, and a four-frames-in-one article that contained the major points of all the four frames (separated by introductory language like "another group of foreign policy experts argues"). To provide viewpoint diversity, our frames differed in key areas of narrative and interpretation according to the central concerns of each value orientation (Baden and Springer 2015). To avoid recency and primacy effects, we created two versions of the four-frames-in-one article: one in which the two Interventionist frames were presented first, and another in which the two Isolationist frames were presented first. We sought to limit, if not eliminate, additional influences as much as possible, so as to analyze framing effects in isolation: No cues were provided about the identity or political party of the "experts" expressing their views, nor were cues provided about the ideological bent or credibility of the media outlet. In addition, to ensure that our results were not skewed by any extraneous factors unique to the Ukraine conflict (like prejudice against Russia or the EU), we created "masked" versions of all six articles, replacing all mentions of the countries and organizations involved with "Country A, B, C" and "Political Coalition D," and so on.

Procedures
The separate group of pretest participants were asked first to describe their perspective on the Ukraine conflict in a few sentences, then to read one of the articles we designed, and finally to rate it in terms of familiarity (how often they have seen this sort of argument before in the media) and how persuasive they found it. Based on our review of media coverage of the Ukraine crisis, we expected the LISO frame to be least familiar, as we found it only in relatively marginal sources like Democracy Now! and The Nation.
Using and combining elements of past scales to measure foreign policy values, we created five multiple-choice questions to ask participants, each of which had four answers corresponding to LISO, LINT, RINT, and RISO foreign policy values (see Online Appendix B). Each participant therefore had five opportunities to define their foreign policy values according to these four orientations. We also included questions asking participants to rate how strongly they held the values they selected, how many articles or TV programs on Ukraine they have been exposed to, strength of their pretreatment opinion about the Ukraine conflict, and a five-item quiz on basic facts about contemporary Ukraine (e.g., bordering countries, current president). Participants were also asked to place their political orientation on a 100-point, left/liberal to right/conservative scale, to rate how helpful the article was in forming their opinion, how many of the four orientations they wanted to read more about on the Ukraine conflict, and standard demographic questions.
Using a between-subjects design, we randomly assigned participants to read either the control article, one of the single-frame articles, or the multiple-frame article, in either Ukraine or "masked" format. After reading their assigned article, participants were asked five multiple-choice questions on the Ukraine conflict, with four answers each corresponding to the arguments of the LISO, LINT, RINT, and RISO frames (see Online Appendix C). These five Ukraine questions allowed us to compare congruence between participants' stated foreign policy values in the abstract, and their preferences for concrete application of values in a real-world situation. We measured congruence between foreign policy values and Ukraine conflict opinions by calculating "change scores," subtracting foreign policy values answers from Ukraine conflict answers in each of the four value orientations (LISO, LINT, RINT, and RISO). (These change scores comprise value orientation responses in related but different questions; also, it is possible that value consistency suggested by lower change scores may be partially due to a desire to seem consistent.) We expected that compared with those receiving single-frame treatments, those receiving the multiple-frame treatment would respond with Ukraine conflict answers that more closely matched their foreign policy values answers (H1 and H2).

Pretest Results
We individually coded participants' open-ended statements as corresponding to the LISO, LINT, RINT, or RISO frame, or whether they did not match any of the four frames. The interrater reliability for the coders was kappa = .788 (p = .001). Averaging between the two coders' results, a majority (63.9 percent) of participants' open-ended statements were unclear or did not match any of the four frames; 13.5 percent corresponded to RINT, 11.7 percent corresponded to RISO, 7.2 percent corresponded to LINT, and only 3.6 percent corresponded to the LISO frame. Similarly, the mean ratings (from zero to ten) for familiarity with and prior exposure to the frames were 5.8 for LINT, 5.3 for RINT, 4.6 for RISO, and only 3.8 for LISO. A one-way analysis of variance (ANOVA) found marginally significant, F(3, 107) = 2.192, p = .093, differences between these four means, and a significant, t(109) = −2.108, p = .037, difference between the mean familiarity for LISO versus mean familiarity for the other three frames as a group. Hence, the LISO frame was the least familiar to pretest participants. However, the mean persuasiveness or strength ratings (from zero to ten) for each of the four frames were between 6 and 7, with a one-way ANOVA finding no significant differences, F(3, 107) = 0.844, p = .473, between them. Each of the four frames were viewed as equally persuasive, regardless of the fact that the LISO frame was significantly less familiar to pretest participants than the other frames.

Main Results
First, to see if factors other than those we intended to test were affecting the results (for instance, a strong identification with Ukraine or Russia, subtle racial/ethnic, partisan, or expertise cues), we tested for differences in level of opinion change between the Ukraine and "masked" conditions. A one-way ANOVA found no significant differences in opinion change between the Ukraine and "masked" conditions, F(11, 544) = 0.663, p = .774. Our further analyses focused only on those receiving the Ukraine treatments.
Next, to see whether single frames shifted opinions in the direction of the frame, we performed one-sample t tests on mean change in the number of LISO, LINT, RINT, and RISO answers to values questions pretreatment and Ukraine conflict questions posttreatment. For instance, a participant who chose four RISO and one RINT answers to the foreign policy values questions and then chose three RISO and two LISO answers to the Ukraine conflict questions after reading the LISO article, would have "change scores" of 2 for LISO (2 − 0), 0 for LINT (0 − 0), −1 for RINT (0 − 1), and −1 for RISO (3 − 4). We expected that exposure to each single frame would occasion a change in opinions away from values and toward the position of the frame.
First, we analyzed the control group: There was no statistically significant movement toward any of the four frames except for RINT, which saw an average of a onehalf answer increase, t(52) = 3.297, p = .002. This could be due to instrument error, for example, a mismatch between the questions for pretreatment foreign policy values and those for posttreatment Ukraine conflict opinions, or due to a relative distaste for the RINT perspective in the abstract, but a greater preference for the application of RINT policies once contextualized in the real world. Regardless, because one of the four orientations demonstrated significant movement pre-and postcontrol treatment, we supplemented our subsequent t tests with regressions that compared change scores in the treatment groups with that of the control group.
All the single frames except LISO provoked movement away from pretreatment values toward the orientation of the frame to which participants were exposed. The group exposed to the LINT frame increased their LINT policy orientation by an average of more than one full answer, t(39) = 4.253, p = .000, the group exposed to the RINT frame increased their RINT policy orientation by an average of just below one full answer, t(55) = 5.601, p = .000, and those exposed to the RISO frame increased their RISO policy orientation by about two thirds of an answer, t(46) = 2.961, p = .005. In other words, whatever the proportion of pretreatment values answers, exposure to single LINT, RINT, and RISO frames provoked a significantly greater average of posttreatment Ukraine conflict answers to match the respective frame. To compare these changes with change in the control group, a dummy variable for each singleframe condition was included in simple linear regressions, with change between answers to (pre) values and (post) Ukraine conflict questions as the dependent variable. The results remained robust and significant at the p < .05 level; with changes in the control group as a baseline, the models predict that exposure to the LINT frame changes slightly above one answer to LINT, the RISO frame changes nearly one answer in its direction, and the RINT frame changes about half an answer in its direction (see Figure 1).
In the LISO condition, however, the opposite effect is evident: those exposed to the LISO frame decrease their Ukraine opinion LISO orientation by just more than half of an answer, t(50) = −2.824, p = .007). However, comparing the effects of exposure to the LISO frame with exposure to the control frame in regression reveals a smaller negative effect of about one third of one fewer LISO answer, but without reaching conventional levels of significance, β = −.329, t(102) = −1.502, p = .295. Nonetheless, the LISO frame proves to be an outlier, producing a small backfire effect away from the LISO orientation. H1 is partially supported by these results: Exposure to single LINT, RINT, and RISO frames provoked change toward the perspective of the frame, while exposure to the single LISO frame caused a small backfire effect. In contrast, and consistent with H2, exposure to the multiple-frame article resulted in no statistically significant movement toward any of the four orientations. Likewise, compared with movement in the control group in simple regression analyses, exposure to the multiple-frames article resulted in no statistically significant movement toward any of the four orientations (for full results, see Online Appendix D).
However, these results could be due to greater polarization among those receiving the multiple-frame treatment, with strong adherents of each set of values becoming even more supportive of their preferred position, and all participants largely cancelling out each other's changes. To explore opinion change across those participants who did and did not receive a frame matching their foreign policy values, we created a dummy variable indicating whether the participant was randomly assigned to read a frame matching their values (this applies to those who chose the same value at least three out of five times; those who chose a more diverse array of policy values were coded as 0 regardless of their treatment group). For those receiving single frames (in either Ukraine or "masked" conditions), we then ran four univariate ANOVAs using this dummy variable as the fixed factor, and change in LISO, LINT, RINT, and RISO scores, respectively, as the dependent variable. For those receiving the multiple-frame treatment, we created dummy variables indicating whether the participant chose a majority of LISO, LINT, RINT, or RISO values, and ran four univariate ANOVAs using these as the fixed factors, with change in one of the corresponding orientation scores as the dependent variable. (Participants in the multiple-frames condition were exposed to all four frames-so all those with a majority in any of the four value orientations were exposed to a frame matching their values.) This allowed us to compare the influence of single frames versus multiple frames across the four foreign policy orientations. Compared with those exposed to single frames, those exposed to multiple frames had their own preferred values less reinforced and were less influenced to adopt values different from their own, in every condition across the board (see Online Appendix F, for full results). Supporting H2, these results indicate that instead of provoking polarization, exposure to multiple frames may tend to moderate opinions (Figures 2 and 3). 1 To explore some of the other influences operating alongside frames, we ran regressions on the change from stated values to opinions on Ukraine policy, including several other variables: age, a "female" dummy variable, education (from 1, some high school, to 6, completed graduate school), income, political ideology (from left/liberal, 0, to right/conservative, 100), self-reported strength of foreign policy values (from 0 to 10), knowledge of Ukraine (total correct of five questions), strength of pretreatment opinion about the Ukraine conflict, self-reported amount of exposure to articles/TV news segments on Ukraine, rating of how helpful the treatment article was in forming one's opinion, and total number of the four perspectives participants reported they wanted to learn more about ("frame diversity desire"). In line with previous framing research, no single factor reliably explained variation across all conditions. Age and media exposure did not reach statistical significance in any condition, and they were removed from models when doing so provided a better fit (for the full models, see Online Appendix F).
Four separate regressions with the same predictors were run on change in LISO, LINT, RINT, and RISO answers, respectively, for those receiving the multiple-frame treatment (again, compared with the control group). The models predict LISO to be a  Note. Circles represent, in ascending order of size, the self-placement of 1-2, 3-4, 4-5, and 6+ participants, respectively, within the foreign policy values quadrants (before), and Ukraine policy opinion quadrants (after). LINT = Leftwing Interventionism; LISO = Leftwing Isolationism; RISO = Rightwing Isolationism; RINT = Rightwing Interventionism. more likely choice for males, β = −.201, t(88) = −2.064, p = .042, and those further to the right/conservative, β = .398, t(88) = 4.044, p = .022, while LINT was predicted to be a less likely choice for the more educated, β = −.339, t(78) = −3.299, p = .001, and those further to the right/conservative, β = −.313, t(78) = −3.097, p = .003. RINT was predicted to be a more likely choice for those with higher levels of education, β = .223, t(92) = 2.051, p = .044, and (at marginal significance) a less likely choice for those with more knowledge of Ukraine, β = −.194, t(92) = −1.691, p = .095. The only statistically significant predictor of change for RISO answers was the rating of how helpful the multiple-frame article was: the more helpful participants found the article, the more likely they would move away from the RISO orientation in their Ukraine policy answers, β = −.223, t(84) = −2.094, p = .039.
Overall, knowledge about Ukraine and strength of adherence to one's foreign policy values had a weak dampening effect on opinion change after reading the treatment articles: those with greater knowledge and confidence stayed somewhat closer to their stated values when answering questions on Ukraine policy. Strength of values and opinion change were weakly, negatively correlated, r(227) = −.092, p = .082. Knowledge about Ukraine and opinion change were also weakly, negatively correlated, r(227) = −.109, p = .051. This provides some support for H3.

Discussion
Our results provided support for H1, that exposure to single frames would move opinions away from foreign policy values and toward the position of the frame. This was the case for LINT, RINT, and RISO frames, and this result held after comparing movement with the control group. However, the LISO frame provoked a "backfire" effect (Nyhan and Reifler 2010), pushing participants away from the positions advocated in the LISO article (although compared with the control group, the movement away from LISO was not significant). Chong and Druckman (2007) found backfire effects for weak frames, but our frames were all of substantially similar strength (as revealed by pretest ratings). However, the LISO frame was substantially less familiar to pretest participants-they had been exposed to it less frequently in the media-so, just as Chong and Druckman (2007: 645) conclude, "Frames that draw on unavailable beliefs (as these frames do according to our pretest) do not affect opinions." These results reinforce two recurring normative concerns with media framing. First, media reports that rely on one frame of an issue influence readers to adopt that frame, regardless of the reader's values. Second, perspectives on an issue that do not commonly appear in media presentations may suffer a handicap, such that when finally exposed to uncommon perspectives, readers may be inclined to reject them due to their unfamiliarity.
The results provided support for H2 that exposure to multiple frames would not significantly move participants' policy opinions away from their values, without causing polarization. As in Sniderman and Theriault's (2004) findings on dual frames, participants stayed closer to their values after reading all four frames on the Ukraine conflict. Also, instead of polarizing opinions (i.e., those with LISO values moving further toward the LISO orientation, etc.), exposure to multiple frames seems to have moderated opinions. Participants receiving the multiple-frame treatment evinced Ukraine policy opinions reflecting not only their own value orientation but also some of the divergent perspectives reflecting other value orientations. Here, competing frames seem to have provided a "knowledge constraint" on the process of motivated reasoning (Kunda 1990). Interestingly, among those exposed to all four frames, the more conservative tended to be persuaded by the LISO orientation while the more liberal tended to be persuaded by the LINT orientation.
For example, imagine a staunch conservative who tends to get her information from conservative media that primarily feature RINT perspectives on foreign policy issues. This conservative, like those of all ideological persuasions, would be liable to engage in motivated reasoning when considering the conflict in Ukraine. Yet when reading an article describing the conflict using four equally strong frames, this conservative would be presented with new information undergirding contrary frames-for instance, the history of NATO expansion or the extreme-right elements in Ukraine's government-which she would not normally be exposed to on a media diet of ideologically congenial single (RINT) frames. This information may pose a "knowledge constraint" to motivated reasoning: It becomes harder to view Russia as an evil aggressor and Ukraine as a good victim when one knows that NATO has continued to expand toward Russia despite earlier promises to the contrary, and that the postcoup Ukrainian government has ties with extreme-right groups. Likewise, information about the antidemocratic nature of the Putin regime in the RINT and LINT frames may pose a knowledge constraint to liberal ideologues who expose themselves disproportionately to the LISO perspective, making them less comfortable with accommodating Russian interests in Ukraine.
H3 received some support from the results: The more knowledge participants had about Ukraine and the stronger participants' foreign policy values, the less influence frames exerted over participants' opinions on the Ukraine conflict. Unlike Lecheler and de Vreese (2011), we found a small, marginally significant negative correlation between knowledge about Ukraine and opinion change after treatment. (This could be because we measured Ukraine-specific knowledge, rather than general political knowledge: One might be well informed about politics generally, but might not have paid much specific attention to the conflict in Ukraine.) This correlation supports the analysis in Beattie (2016) that the more information one has on a topic, organized into schematic structures, the less likely that additional information will change one's opinion at the point of exposure. (However, our experiment only measured opinion change at one point in time; Lecheler and de Vreese's (2011) findings would suggest that over time, those with medium levels of knowledge about Ukraine would display more persistent framing effects, while those with high and low levels of knowledge would be less affected.) Also, we found a small, marginally significant negative correlation between strength of adherence to one's foreign policy values and opinion change after treatment, in accordance with Hansen's (2007) results.
Our sample differed in several respects from a nationally representative random sample, and to this extent, our results are not broadly generalizable. There are additional questions about external validity, particularly in light of the tendency of partisans to choose media sources that reflect their ideology. In practice, it is more likely that news seekers would encounter single frames from ideologically congenial outlets than from those of a different political persuasion, and they would be very unlikely to encounter multiple-frame articles of the sort used in the present experiment. This goes to the core of the News Diversity standard: that the norms of professional journalism should, but do not presently, include the directive to present several ideologically diverse perspectives in each news article. Therefore, the present study should be considered only a first step in exploring the consequences of adopting such a standard. Furthermore, this design does not address practical constraints (column inches, airtime) news companies face. Also, we tested an unobtrusive issue not well known to most participants; tests of issues about which readers already have welldeveloped, extensive schemas are less likely to evince opinion change-especially at one point in time, though possibly over longer periods as well. However, this remains untested, and may be a worthwhile direction for future research. Likewise, opinion responses may have been partially primed by the treatment articles, reflecting only a transitory, unstable opinion-which future research measuring opinion effects over time across single and multiple frames may be able to clarify.

Conclusion
These results reinforce Mauro Porto's (2007a) argument for a "News Diversity" standard: that the media in democracies should take care to provide multiple, equally strong frames for any given issue. This would help avoid the normative concerns surrounding framing effects, whereby single frames influence people to adopt opinions out of line with their values, without causing greater ideological polarization. Even if one views ideological polarization in a positive light, as the result of individuals merely following and deepening their commitments to their own values, single frames only allow those whose values match the frame to follow and deepen their commitment to their own values. Those with different sets of values are poorly served by single frames that contradict their values; only when multiple, equally strong frames are provided are those of all value persuasions well served. Audiences can choose to ignore perspectives they disagree with, but it is harder for them to come up with perspectives on their own, particularly when such perspectives require information the media do not provide (Beattie 2016;Entman 1989). This is particularly clear in the case of the LISO frame, which requires information regarding U.S. support for the coup that brought in the pro-Western regime, that regime's alliance with far-right elements, U.S. corporate interests in the region, the danger of nuclear war, and so on, for the LISO frame to be coherent. Without exposure to the information (whether accurate or not) comprising the LISO perspective, it would be difficult, if not impossible, for audiences to develop a LISO perspective on their own.
Of course, if any given real-world occurrence x is most accurately and fairly communicated by frame A, and across-the-board less accurately and fairly communicated by frames B-Z, then normatively we would prefer for all media outlets to communicate the occurrence of x using only frame A. Mere agreement between less accurate frames B to Z and people's value commitments would seem less normatively important than the agreement between frame A and the real-world occurrence of x the frame is meant to communicate. However, in the absence of well-intentioned expert agreement on the across-the-board superiority of frame A versus frames B through Z-a condition we would expect to obtain in most, if not all, instances of complex, politically charged events-we are left in the epistemically conservative position of preferring a variety of competing frames, holding on to Oliver Wendell Holmes's hope that the "best test of truth is the power of the thought to get itself accepted in the competition of the market" (Abrams 1919: 630).
Although there is a dearth of research into the number of frames commonly used by U.S. media outlets, Benson's (2009) study found fewer frames in United States compared with French media outlets-despite strong theoretical reasons to expect U.S. media outlets to be more "multiperspectival" (Hallin and Mancini 2004), that is, to present more frames. Overall, there is a need for further research into the number of frames commonly used in media reports, as well as whether they are equally strong and presented without preference. Also, in future framing studies, greater ecological validity can be obtained by selecting real-world media reports with greater and lesser frame diversity and equality to test for framing effects.
The "News Diversity" standard-that all media reports should provide multiple, competing, equally strong frames-may avoid the normative concerns that arise from single-framed media reports. Multiple frames at least provide audiences with a choice of perspective on political issues, allowing them to form opinions in closer accordance with their values-or, moderating their opinions after hearing arguments arising from different sets of values or factual information. Future research is needed to test for the effects of multiple, competing frames on a variety of other political issues (we would expect broadly similar results), and on the determinants of frame influence across different issue areas. Furthermore, the current trend in framing research to test for overtime effects is a promising development, and would benefit from including more than two competing frames per issue.

Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The authors received no financial support for the research, authorship, and/or publication of this article.