{'fraction_non_alphanumeric': 0.019438444924406047, 'fraction_numerical': 0.004319654427645789, 'mean_word_length': 5.289592760180995, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Fine-Grained Analysis of Diversity Levels in the News

ABSTRACT Many researchers consider the presentation of diverse content as a prerequisite for the news media to fully exercise their democratic mandate. While prior news diversity studies have contributed important theoretical insights, we argue here that scholarly knowledge of this concept can be significantly advanced by employing computational methods for text analysis. Using automated methods, researchers can increase both the scope of data being analyzed and the resolution of the analysis. This article presents a novel framework for analyzing news diversity consisting of two distinct stages. In the first stage, a computational text classification method is used to analyze, at a high resolution, the attention given in news texts to a broad range of political and social issues. In the second stage, the text classifications are aggregated, and the distributions of media attention to those issues (i.e., news diversity) are assessed on a large scale. After presenting the novel approach, we illustrate its usefulness for testing theoretical hypotheses about news diversity. We compare the diversity of economic coverage in three elite and three popular US newspapers (N = 252,807 articles) and find that a fine-grained analysis relaxes concerns raised in previous studies about low content diversity in the popular press.