{'fraction_non_alphanumeric': 0.028023598820058997, 'fraction_numerical': 0.0029498525073746312, 'mean_word_length': 5.112612612612613, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Theory of Information Compression: When Judgments Are Costly

A Theory of Information Compression: When Judgments Are Costly How useful to tourists are thousands of reviews of different five-star hotels in a city on a travel website when the mean rating is 4.5, and all the five-star hotels score around the mean? How insightful are reviews of physicians on a physician review website to potential patients when the ratings cluster tightly around an average for all physicians? Are there costs to the physicians, the patients, and to society as a whole? When all the students at a university score “A” grades on most courses, are there consequences for the university, the students, and potential employers? This paper calls the “clustering around a mean” phenomenon “information compression” and the systems in which it occurs (e.g., universities, students, employers) “judgment networks.” When there is extensive information compression in a system, measures such as ratings or grades have little value for decision makers. When all five-star hotels in a city score an average of 4.5 does it really matter which one a traveler chooses? The paper introduces a way of measuring information compression. It also suggests ways for organizations to overcome the negative consequences of information compression for themselves and their various stakeholders.