{'fraction_non_alphanumeric': 0.021700879765395895, 'fraction_numerical': 0.0017595307917888563, 'mean_word_length': 5.295202952029521, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Indexing versus modeling intercoder reliability

Although it has long been a consensus that the intercoder reliability test is crucial to the validity of a content analysis study, the choice among them has been debated. The significance of this dissertation was first demonstrated through examining how communication scholars have reported intercoder reliability in content studies. Following that, the theoretical framework on which all the intercoder reliability approaches are based was critically reviewed. The origins of these approaches and connections among them were explored. After that, I reviewed extant 33 intercoder reliability indices and empirically tested most of them, aiming to find the most robust index against prevalence and rater bias, by testing their relationships with the response surface methodology through a Monte Carlo experiment. It was found that Maxwell’s R.E (or the S family) is superior to λ of Goodman and Kruskal, Krippendorff’s α, Scott’s π, Cohen’s κ, Ir of Perreault and Leigh, and Gwet’s AC1 in the case of two-coder and binary outcomes. More nuanced relationships among prevalence, sensitivity, specificity and the intercoder reliability indices were discovered through response surface plots. The similarities as well as distinctions among these indices were further investigated via multiple comparisons. In another study also based on Monte Carlo simulation, S is found to be only affected by the number of categories. Consequently, S is a category based index. The chance agreements of α, π and κ are affected by the marginal distribution, the level of difficulty and the interaction between them. The three indices are in general distribution based indices.