{'fraction_non_alphanumeric': 0.040336134453781515, 'fraction_numerical': 0.0016806722689075631, 'mean_word_length': 5.203125, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reliability of Total Test Scores When Considered as Ordinal Measurements

This article studies the ordinal reliability of (total) test scores. This study is based on a classical-type linear model of observed score (X), true score (T), and random error (E). Based on the idea of Kendall's tau-a coefficient, a measure of ordinal reliability for small-examinee populations is developed. This measure is extended to large (theoretically, infinite) examinee populations. This extended measure is identified as the population version of Kendall's tau-a. Appropriateness of the traditional product moment correlation-based measures,. TX and .XX0, is reviewed. It is observed that. TX and .XX0 are appropriate within the family of normal distributions of (T, E) but are inappropriate in the family of nonnormal distributions. A simulated data example is considered to show that the traditional product moment correlation-based measures may be misleading. Furthermore, practical procedures are suggested for estimating ordinal reliability. Real test score data are used for demonstration. Index terms: ordinal measures, linear model in classical test theory, test score reliability, Kendall's tau-a