{'fraction_non_alphanumeric': 0.023430962343096235, 'fraction_numerical': 0.0008368200836820083, 'mean_word_length': 5.328042328042328, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
6. Statistical Analysis and Power Estimation

Often the question of most interest to the experimenter is whether Modality A is better than Modality B, with performance measured by receiver operating characteristic (ROC) area, Az. Since Az estimates are subject to statistical fluctuations, a simple numerical comparison is not sufficient, because there are uncertainties in the Az estimates for the two modalities, and hence in their difference, that must be taken into account. In general, an estimate of whether an observed difference is due to a real difference in the modalities or is due to chance alone can be calculated from the observations by using the method of hypothesis testing. A null hypothesis is formulated stating that the two modalities are identical in performance. The statement that they are different in performance is called the alternative hypothesis. Clearly, the larger the observed difference (DAz) between two modalities, relative to the inherent variability, s(DAz), the smaller the probability that a chance fluctuation produced the observed deviation. Hence, a fundamental quantity in statistical hypothesis testing is the so-called critical ratio (CR) defined by