{'fraction_non_alphanumeric': 0.02092050209205021, 'fraction_numerical': 0.008368200836820083, 'mean_word_length': 5.214285714285714, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Statistical inference of Gwet’s AC1 coefficient for multiple raters and binary outcomes

Abstract Cohen’s kappa and intraclass kappa are widely used for assessing the degree of agreement between two raters with binary outcomes. However, many authors have pointed out its paradoxical behavior, that comes from the dependence on the prevalence of a trait under study. To overcome the limitation, Gwet (2008) proposed an alternative and more stable agreement coefficient referred to as the AC1. In this paper, we discuss a likelihood-based inference of the AC1 in the case of multiple raters and binary outcomes. Construction of confidence intervals is mainly discussed. In addition, hypothesis testing, sample size estimation, and the method of assessing the effect of subject covariates on agreement are also presented. The performance of the estimator of AC1 and its confidence intervals are investigated in a simulation study, and an example is presented.